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What Is NAEP? 



T he National Assessment of Educational Progress (NAEP) is a 
congressionally mandated project of the U.S. Department of 
Education’s National Center for Education Statistics. It assesses 
what U.S. students should know and be able to do in geography, reading, 
writing, mathematics, science, U.S. history, the arts, civics, and other 
academic subjects. Since 1969, NAEP has surveyed the achievement of 
students at ages 9, 13, and 17 and, since the 1980s, in grades 4, 8, and 12. 

Measuring educational achievement trends over time is critical to 
measuring progress toward the National Education Goals. 

The National Assessment Governing Board 

The National Assessment Governing Board (NAGB) was created by 
Congress to formulate policy for NAEP. Among the Board’s responsibili- 
ties are developing objectives and test specifications and designing the 
assessment methodology for NAEP. 
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Introduction 



T his document contains the framework and rationale for 
assessing science achievement of students throughout the 
United States in 1996 and 2000. It provides a general over- 
view of the National Assessment of Educational Progress (NAEP), 
describes the NAEP Science Framework adopted by the National 
Assessment Governing Board (NAGB), and reviews the process by 
which the Framework was developed. 

Background 

NAEP is authorized by Congress and funded by the federal 
government; it is the only nationally representative and continuing 
assessment of what America’s students know and can do. For 30 
years, NAEP has been charged with collecting and reporting infor- 
mation on student achievement in mathematics, reading, science, 
U.S. history, writing, and other subjects. NAEP assessments were 
conducted on an annual basis until 1981, when they became bien- 
nial. Originally, assessments were of students at ages 9, 13, and 17, 
but beginning in 1983, they have also included students in grades 4, 
8, and 12. 

NAEP reports provide descriptive information about student 
performance — including basic and higher order skills — in various 
subjects and comparisons of performance by race/ethnicity, gender, 
type of community, and geographic region. They also show rela- 
tionships between achievement and certain background variables, 
such as the time spent on homework or the educational level of 
parents. 

In the past, only results from a national sample of students were 
reported for each NAEP subject assessed, but in 1987 a national 
study group chaired by Lamar Alexander — then Governor of 
Tennessee — and H. Thomas James — President Emeritus of the 
Spencer Foundation — recommended to the Secretary of Education 
that NAEP collect representative data on student achievement at 
the state level. In 1990, a trial state assessment was conducted for 
8th-grade mathematics. The trial continued in 1992 with state-level 
assessments in mathematics at grades 4 and 8 and reading at grade 
4. In 1994, reading was again assessed at the state level in grade 4. 
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State-level assessments have continued in reading, writing, math- 
ematics, and science. 

Assessment is not without critics. The Alexander-James Study 
Group expressed concern that “national assessments of science 
were sporadic and almost exclusively devoted to assessing factual 
information,” and that they did not attempt to assess abilities to 
organize and transform a body of information into a coherent 
scientific account {The Nation’s Report Card: Improving the 
Assessment of Student Achievement, 1987). 

To address this criticism, the study group recommended that 
NAEP broaden its scope to include the collection of information 
on whether students are able to design, perform, and analyze 
experiments; on whether they have acquired complex thinking 
abilities essential to various fields of science; on whether they are 
able to integrate basic concepts of the various scientific fields; and 
on whether they can perceive fundamental relationships. To mea- 
sure these kinds of knowledge and abilities, NAEP was urged to 
include open-ended items and performance tasks in its assessment 
techniques. 

NAEP has great prestige, considerable influence, and well- 
publicized results. As a result, the weight of its findings may have a 
substantial impact on science education, possibly influencing state 
curriculum frameworks and, ultimately, even what teachers teach. 
Therefore, the assessment should reflect consensus on priorities, 
best practices, and conclusions from research on science education. 

A specific example of how NAEP can influence state frameworks 
is illustrated by the selection of special assessment techniques, such 
as hands-on science. Performance, for instance, places a value on 
the “doing” of science. That priority actually may not be reflected 
in classroom practice, although it always was considered a standard 
teaching technique in science. Thus, as NAEP science reports are 
circulated, states and curriculum policymakers become sensitized 
and may incorporate more hands-on activities into classroom lessons. 

Science educators, by and large, do not quarrel with the essential 
concept of assessment, but there has been no formal agreement on 
a common framework, outcomes, or goals and objectives to assess. 
The NAEP Science Assessment attempts to reflect a comprehen- 
sive, contemporary view of science so that those affected by the 
National Assessment are satisfied that they address the complex 



issues in science education without oversimplification. The framers 
of this document have attempted to tread a fine line between clear 
communication and technical accuracy in the hope that their efforts 
represent a step forward in building national consensus on the kev 
outcomes of science education. ^ 



Development of the 1996 and 2000 NAEP 
Science Assessment Framework 



The following factors guided the process for developing consen- 
sus on the NAEP Science Assessment Framework: 



The first factor is the general process of consensus develop- 
ment, both as it is set forth in law and as it evolves over time. 

The process calls for active participation and broad involvement of 
curriculum specialists; science teachers; local science supervisors; 
state supervisors; administrators; parents; and representatives of 
scientific associations, business and industry, government, and 
unions; cognitive psychologists; and science educators, as well as 
participation from the public and private education sectors. 

The second factor is emphasis on the important outcomes of 
science education. As much as possible, this Science Framework 
represents what is considered essential learning in science, which 
set the stage for the 1996 and subsequent assessments. The Frame- 
work also recommends innovative assessment techniques to probe 
critical abilities and content areas. 



The third factor is recognition that the various “players” 
m education and industry often hold diverse and sometimes 
Views. Further, research and general agreement in the 
field IS lacking. This lack of agreement on a common scope of 
instruction and sequence, components of scientific literacy, impor- 
tant outcomes of learning, and the nature of overarching themes in 

science mders clear communication between science educators 
and the public. 



The process of developing the Framework document and accom 
panymg reports occurred between October 1990 and August 1991 
Original plans called for the new NAEP Science Assessment to be 
given m 1994. Due to a budget shortfall, however, both the new 
science and mathematics assessments were rescheduled for 1996. 
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For the consensus process, 3 steering committee of 19 members 
(see appendix E) recommended by the education community and 
private sector was established. Its members developed the princi- 
ples that guided the creation of the Framework. A smaller planning 
committee (see appendix E), composed of practitioners, recognized 
experts in science education, and scientists, was established to iden- 
tify goals and objectives and to produce the Framework. Together 
with the steering committee, it was also responsible for suggesting 
ways to assess important outcomes of science education. 

The planning committee met monthly from November 1990 
through April 1991 and was joined in the first and final meetings 
by the steering committee, which reviewed and reacted to all 
Framework drafts. Staff of the American Institutes for Research 
(AIR), a subcontractor, also attended the meetings and, on the 
basis of this input and with reaction and advice from the commit- 
tees, AIR formulated specifications for the science assessment. 

The NAGB Subject Area Committee #2 and technical staff 
closely monitored the consensus project work, and Board members 
were involved in all key phases. In addition, advice was sought 
from the organizations and sectors that affect and are affected by 
science education. For instance: 

• Opportunities for public input were provided at two hearings 
(in Washington, DC, and San Francisco). Opinions were 
expressed by representatives of institutions (for example, 
public and private education and scientific and science educa- 
tion organizations) likely to be affected by or concerned with 
the Framework and subsequent assessments. 

• The Council of State Science Supervisors was kept abreast of 
project developments through its electronic communication 
system, PSInet. This was vital, because state science instruc- 
tion as well as public opinion may be influenced by the 1996 
and 2000 assessment processes, state policymakers and those 
with responsibility for science education leadership must 
become familiar with and involved in shaping the process 
and products of the science consensus effort. Input from the 
Council of state Science Supervisors was particularly impor- 
tant in helping the NAEP Science Assessment Project define 
“big ideas” or themes in science learning (see chapter two). 
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For example: 

— States with existing frameworks for science instruction 
were contacted for copies of their frameworks and assess- 
ment methods. 

Planning and steering committee members hosted sessions 

at both regional and national science education meetings 
to report on the development of the Science Assessment 
Framework. 

^The revised draft of the Framework was widely circulated 

in June 1991 within the science education and science 
communities for reaction and comments. 

Steering Committee Guidelines 

At its early meeting, the steering committee drafted guidelines 
for the planning committee’s work and recommended that the 
Framework and ensuing NAEP Science Assessment have the 
following five characteristics: 

• The Framework should reflect the best thinking about the 
knowledge, skills, and competencies needed for a high degree 
of scientific understanding among all students in the United 
States. Accordingly, it should: 

— Encompass knowledge and use of organized factual infor- 
mation, relationships among concepts, major ideas unifying 
the sciences, and thinking and laboratory skills. 

— Be based on current understandings from research on 
teaching, learning, and student performance in science. 

• Both the Framework and the 1996 and 2000 NAEP Science 
Assessment should: 

— Address the nature and practices of knowing in science, as 
different from other ways of knowing. 

— Reflect the quantitative aspects of science as well as the 
concepts of the life. Earth, and physical sciences. 

— Deal with issues raised by the role of science and technology 
in society. 
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Include practical problem solving that involves design, use 
of materials, and weighing risks in relation to benefits.' 

Take into account the developmental levels of students. 

— Ensure that students with diverse backgrounds are assessed 
in ways that provide them with equal and fair opportunities 
to reflect their knowledge and performance. 

• Assessment formats should be used that are consistent with 
the objectives being assessed. A variety of strategies for 
assessing student performance are advocated, including; 

Performance tasks that allow students to manipulate phy- 
sical objects and draw scientific understandings from the 
materials before them. 

Open-ended items that provide insights into students’ levels 
of understanding and ability to communicate in the sci- 
ences, as well as their ability to generate, rather than simply 
recognize, information related to scientific concepts and 
their interconnections. 

Collections of student work over time (such as portfolios) 
that demonstrate what students can achieve outside the time 
constraints of a standardized assessment situation. 

—Multiple-choice items that probe students’ conceptual 

understanding and ability to connect ideas in a scientifically 
sound way. 

• The assessment should contain a broad enough range of items 
at different levels of proficiency for identifying three achieve- 
ment levels for each grade. 

• Information on students’ demographic and other background 
characteristics should be collected. Additional information 
should be collected from students, teachers, and administra- 
tors about instructional programs and delivery systems so 
that their relationships with student achievement can be ascer- 
tained and used to inform program and policy decisions. 
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Chapter One 



The Nature of Science and the Science 
Curriculum 



The Nature of Science 

T he various fields of science have their own special ways of 
knowing, but the essentials of the natural sciences should be 
defined for the purpose of planning appropriate assessments. 

The natural sciences are characterized by organized explanations 
incorporating both theoretical and empirical elements. Scientists 
attempt to construct theories that encompass as much factual and 
conceptual knowledge, including laws and principles, as possible. 
The construction of theories is a process involving the consider- 
ation of factual evidence, insightful questioning, creativity, and 
imagination. 

Atomic theory and evolution theory are good examples of mod- 
em scientific theories that are central to the sciences as ways of 
knowing. They are sources of new. hypotheses and logical deduc- 
tions that can be tested. As these theories are refined, they continue 
to stimulate new questions and hypotheses. As with other scientific 
theories, each time they come into play in experimental situations, 
they are again subject to testing and the possibility of refutation. 
Each success broadens their domain and increases their usefulness. 
Verification of theories does not ensure truth, but it does extend 
their usefulness in explaining natural phenomena. 

Useful theories enable an individual to make predictions under 
a specified range of conditions. Scientific theories must be testable 
according to standards of evidence and logical argumentation set by 
the scientific community; they are central to the scientific way of 
knowing and guiding observation. When scientists observe natural 
phenomena, their observations are made within the context of exis- 
ting scientific theories. Nonscientists holding different views may 
perceive the same phenomena, but, through their subjective belief 
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systems, may arrive at different conclusions. History is replete with 
examples of misunderstandings and miscommunication that are 
based, at least in part, on the use of different rules of observation. 

Science consists of both theoretical and experimental knowledge 
that is constructed by the creativity, knowledge, and world view 
of scientists. Experimentation plays an essential role in generating 
and verifying scientific information. New findings and results of 
scientific observation and experimentation accumulate, enlarging 
the base of present knowledge and laying the foundation for future 
learning. Usually, scientific knowledge is structured, forming a web 
of interrelated concepts, laws, and principles. Whereas sensory data 
are sometimes ephemeral, concepts are precisely formulated and 
are multiply connected to other concepts or sensory data within a 
theory. Concepts without factual content (sensory data) are empty, 
whereas sensory data without concepts are difficult to understand 
and open to misinterpretation. 

None of the processes usually associated with science — for 
example, observing, measuring, classifying, deducing, inferring — 
is unique to science. However, in science, these processes are given 
meaning by the context of the subject matter under investigation. 
Hence, observations of a mealworm seemingly walking endlessly 
around the sides of a container have new meaning when mealworm 
anatomy, behavior, and physiology are understood. Likewise, the 
Wiggly lines a physicist observes on a photographic negative taken 
of a cloud chamber remain wiggly lines to most observers, but 
they become important sensory data (facts) when the conditions 
under which the lines were produced are known. A goal of science 
education, therefore, is to help students recognize the difference 
between personal opinion and knowledge gained through scientific 
investigation and debate. 

Although the scientific disciplines are alike in their reliance on 
evidence, use of logic, and organization of factual information into 
concepts and theories, they differ with respect to what constitutes 
evidence, specific methods of investigation, and degree of quantifi- 
cation. Yet there are common understandings among [scientists] 
about what constitutes an investigation that is scientifically valid” 
(American Association for the Advancement of Science, 1989). 

This view of the nature of science has profound implications for 
assessment, as reflected in the criteria outlined in chapter three. 
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The Science Curriculum 



Because a major purpose of NAEP is to illuminate education 
policy, assessment of student science learning must take into 
account the science curriculum. Hence, the 1996 and 2000 NAEP 
Science Assessment Framework is based on a consensus regarding 
desirable elements of science education against which student 
attainment is to be measured. 

In developing the Framework, the planning committee reviewed 
key blue-ribbon reports, examined exemplary practices, studied 
local and state-based innovations in science curricula, reviewed 
science education literature, and noted innovations emerging in 
other countries. Recent reports by government agencies and profes- 
sional societies (National Science Board, 1983; American Associa- 
tion for the Advancement of Science, 1989; National Science 
Teachers Association, 1989; National Research Council, 1990) 
express unanimity in their goals for science education. For example: 

• Students should acquire a core of scientific understanding, 
including organized factual information. 

• Students should acquire the ability to relate scientific concepts 
to one another and to problems the students encounter in and 
out of school. 

• Students should be able to apply science knowledge in 
practical ways. 

• Students should be familiar with experimental design and 
have the ability to carry out scientific experiments that are 
developmentally appropriate. 

• Students should acquire the science knowledge and under- 
standing that will allow them the opportunity to pursue further 
study in scientific fields or enter science- or technology- 
related careers. 

There also are similarities in the reports’ recommendations for 
the science curricula and instruction needed to achieve these goals, 
including: 

• Reduction of the traditional breadth of coverage in favor of 
greater depth, especially in high school science. 
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• Emphasis on development of such thinking processes as 
organizing factual knowledge around major concepts, defining 
and solving problems, accessing information and reasoning 
with it, and communicating with others about one’s science 
results and understandings. 

• Multidisciplinary and interdisciplinary approaches to science 
teaching. 

• Approaches that encourage active student involvement and 
participation, such as participating in hands-on science acti- 
vities; learning in small, cooperative groups; reflecting orally 
and in writing upon experiences; and completing sustained 
projects. 

• Increased participation of underrepresented populations in 
challenging school science. 

The design of the new 1996 and 2000 NAEP Science Assessment 
Framework, while maintaining some conceptual continuity with the 
1990 NAEP Science Assessment, takes into account current reforms 
in science education. It is also consistent with the science frame- 
work used for the 1991 International Assessment of Educational 
Progress and the science framework proposed for the International 
Association for the Evaluation of Educational Achievement (lEA) 
assessments planned for 1995. This consonance is important as the 
nation monitors its progress toward the National Education Goals 
through the year 2000. 
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Chapter Two 







The Framework for the 1996 and 2000 
NAEP Science Assessment 





Introduction 

I t is customary to collect relevant curriculum guides, frameworks, 
and other course outlines to get a sense of what students are 
studying throughout the nation when developing a framework for 
an educational assessment. The union of all these “learning guides” 
is used to develop overall test specifications. Such an approach 
tends to have an unfortunate consequence: It often leads to a broad, 
trivialized, lowest common denominator approach to assessment. 
However, these materials can serve another purpose — documenting 
trends and developments in science education throughout the 
country. In the NAEP Science Assessment Framework development 
process, science reform reports from states and some large-city 
science curriculum guides were gathered and used to establish 
consonance between the evolving framework and the most forward- 
looking of the reports and guides. 

The traditional approach to teaching science tends to emphasize 
rote memorization of facts without connection or organization. 
Although it must not lose sight of the need for factual knowledge 
that is fundamental to science literacy, the assessment of science 
achievement must change to give more emphasis to conceptual 
understanding and the application of knowledge and skills for 
several compelling reasons. First, the expansion of scientific infor- 
mation has resulted in far too many facts for a student to memorize. 
It is more efficient to store them electronically (or in other forms) 
and access information as needed. Second, isolated science facts 
that are not organized or used tend to be forgotten quickly and, 
even when remembered, form a poor basis for learning. Third, it 
is desirable to encourage science instruction that is used both to 
deepen understanding and to address challenging problems. Science 
education is best served when students can understand and discuss 
ideas rather than simply accumulate unconnected facts. 
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Therefore, it is important that science assessment covers major 
topics like electricity and magnetism, forces and motions, life 
cycles, ecosystems, plate tectonics, and climatology. But even these 
need to be viewed as shorthand for a much richer understanding of 
what students should attain. Several current reform reports and 
frameworks, such as Science for All Americans (American Associa- 
tion for the Advancement of Science, 1989), innovative state 
frameworks, and the reports of the National Center for Improving 
Science Education (1990, 1991) describe desired outcomes of 
science instruction in new ways. They advocate mastery of funda- 
mentals in ways that are more likely to result in students’ learning. 
The approach advocated in these documents is reinforced by the 
findings of science educators and cognitive researchers, demon- 
strating that if students do not completely learn the concepts 
presented to them, they pass through the K-12 grades without 
fundamentally changing the conceptual models they learned in 
their early years. 

While it is easy to argue for a test that emphasizes conceptual 
understanding rather than topical listings and the recognition of 
definitions from a list of choices, it is more difficult to agree on how 
science learning should be assessed. The science courses that were 
developed in the late 1950s and early 1960s used a more conceptual 
organization for textbooks and teaching materials. These programs, 
funded by the National Science Foundation, became major influ- 
ences on the country’s secondary science textbooks from 1960 to 
1970 and helped raise standards of students’ science achievement 
(Shymansky et al., 1983). Unfortunately old-style textbooks have 
since returned, with expanded numbers of pages that include many 
newly emerging topics. The effect on students’ understanding of 
science has not been beneficial. For 1996 and 2000, the NAEP 
Science Assessment Framework incorporates a balance of knowl- 
edge and skills based on current reform reports, exemplary curricu- 
lum guides, and research on the teaching and learning of science. 

Framework Elements 

The Framework for the 1996 and 2000 NAEP Science Assess- 
ment is organized according to two major dimensions: Fields of 
Science and Knowing and Doing Science. The Fields of Science 
are the Earth, physical, and life sciences. Knowing and Doing 
Science includes conceptual understanding, scientific investigation. 



and practical reasoning. The two dimensions and subdimensions are 
explained in greater detail below and in appendix B. 

The Matrix 

The NAEP Science Assessment Framework is structured acc- 
ording to a matrix similar to that used for the 1990 NAEP Science 
Assessment. The content areas are organized into the same three 
fields, however, an additional requirement for some interdiscipli- 
nary exercises merges technology with the science content areas. 
The Nature of Science (which also was part of the 1990 Frame- 
work) and Themes (which is new to the 1996 and 2000 NAEP 
Science Assessment) are categories that should integrate the three 
fields of science, rather than represent separate content. The Know- 
ing and Doing dimension is a reorganization of the Thinking 
Skills dimension that was a part of the 1990 Science Assessment, 
with a clearer delineation of subcategories, particularly with re- 
spect to Practical Reasoning. Each element of the 1996 and 2000 
Science Assessment Framework is addressed briefly below and in 
greater detail in separate appendices. 



Fields of Science 



Knowing 
and Doing 

Conceptual 

Understanding 



Scientific 

Investigation 



Practical 

Reasoning 



Earth Physical Life 





















Nature of Science 


Themes 

Models, Systems, 
Patterns of Change 
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With respect to Fields of Science, the main emphasis of the 
assessment should be on knowledge in the content areas. 

• Distribution of content across the three science fields should 
be approximately equal in grades 4 and 12. 

• For grade 8, the Assessment Framework places a somewhat 
heavier emphasis on life science (40 percent), with physical 
and Earth sciences distributed equally (30 percent each). The 
distribution for grade 8 reflects the importance of human 
biology for this age group, which gets increased recognition 
in both curriculum and instruction. 

• A limited number of exercises at every grade level should 
address technology and its relationship to science. Although 
every item need not do so, exercises and tasks that draw from 
more than one discipline at the same time are highly desirable, 

as they are more likely to mirror science problems that occur ' 
in the real world. 

More specific guidance on the distribution of content from the 
three fields is given in the Specifications document. 

A major emphasis with respect to Knowing and Doing Science 
should be on students’ active expression of conceptual understanding. 

• At each grade level, 45 percent of content should be devoted 
to Conceptual Understanding— the ability to understand 
basic concepts and tools used in the process of a scientific 
investigation. 

Scientific Investigation — the ability to use the appropriate 
tools and thinking processes in the application of science- 
should be more heavily emphasized in grade 4 (45 percent) 
than in grades 8 and 12 (30 percent at each of these grade 
levels). This is desirable because learning by doing is crucial 

for younger students, and ways of knowing in science need to 
be introduced early. 

Practical Reasoning involves suggesting effective solutions 
to everyday problems by applying scientific knowledge and 
using skills. The ability to engage in practical reasoning is 
essential if a student is to solve complex problems or apply 
previous knowledge to an everyday problem. The proportion 
of practical reasoning questions should be 10 percent in grade 
4 and 25 percent in grades 8 and 12. 

21 
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• Many exercises will involve more than one of the subdimen- 
sions in Knowing and Doing Science. 

• The percentages cited above should not be interpreted as 
immutable but should serve as a general guide for test devel- 
opment. As new assessment tasks are developed, assessment 
of conceptual understanding may become a part of exercises 
that measure scientific investigations and practical reasoning. 

More detailed guidelines on the distribution of Knowing and 
Doing items are provided in the Specifications document. 

Every question or task in the assessment should be classifiable 
by one or more subcategories in each of the two major dimensions 
of the matrix. In addition to the two major dimensions, the Frame- 
work includes two other categories, which pertain to a limited 
subset of items. The first concerns students’ understanding of the 
Nature of Science. 

• Included in this category are the history of science and 
technology, the habits of mind that characterize these fields, 
and methods of inquiry and problem solving. 

• At least 15 percent of the content should measure the Nature 
of Science. Within this percentage, somewhat more than half 
(about 60 percent) should deal with the nature of science and 
somewhat less than half (about 40 percent) with the nature of 
technology. 

In specifying these percentages, it is assumed that these assess- 
ment items can be developed to measure knowledge of content 
within a field of science or an area of Knowing and Doing 
Science, as well as the Nature of Science. 

The second category. Themes, is new to the 1996 and 2000 
NAEP Science Assessment. Themes represent big ideas or key 
organizing concepts that pervade science education. They cross 
traditional science discipline boundaries and make up the inquiry 
tools that scientists use to better investigate and understand phe- 
nomena. These themes include the notion of Systems and their 
application in the disciplines; Models and their function in the 
development of scientific understanding and application to practical 
problems; and Patterns of Change as they are exemplified in 
natural phenomena. 
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• The assessment should probe, in a developmentally appropri- 
ate way, students’ understanding of these themes. 

• Students in grade 4 should build beginning notions related to 
systems, models, and patterns of change; about one-third of 

the assessment — spread evenly across the three themes 

should measure themes as well as content from one or more 
of the fields of science. 

• Fifty percent of the assessment content in grades 8 and 12 
should assess students understanding of the themes; Questions 
should be distributed evenly among the three themes. 

• The assessment exercises must address a specific theme 
within a science content area so that an understanding of the 
content and the theme are probed at the same time. 

More detailed guidelines on assessing themes are provided in the 
Specifications report, which is used by test developers to create the 
assessment. 

Understanding, doing, and using science often involve tasks that 
include more than one category in each dimension. Multiple-duty 
exercises may present some scoring challenges. Relatively simple 
exercises will be scorable according to one or two subdimensions 
that include, for example, a “conceptual understanding in the phy- 
sical sciences. More complex items may be scorable in several 
subcategories; for example, an open-ended task involving ecosys- 
tems might yield responses scorable according to “conceptual 
understanding in the Earth sciences,” “scientific investigation in the 
life sciences,” and “systems.” Such items, which contribute 
to more than one subcategory, may prove difficult to reproduce 
through similar items having the same properties in future assess- 
ments. It is important, therefore, to specify carefully the several 
subdimensions or subcategories that each such exercise is intended 
to probe. Scoring rubrics for each of the subdimensions also must 
be developed. 



The Fields of Science 

The descriptions below are summaries of the major topic areas to 
be probed within each field in the NAEP Science Assessment. This 
content represents key elements in science that all students should 
be expected to know and understand. For a complete explanation. 
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including descriptions of subject matter knowledge to be expected 
at each grade level, see appendix A. 

Earth Science 

The 1996 and 2000 NAEP Science Assessments will probe 
student understanding of how Earth scientists depict data through 
maps and other means to interpret objects, their features and struc- 
tures, and the events and processes that caused them. What do 
students know about their own position with respect to objects 
and structures on, below, and above the Earth’s surface? What do 
students know about the changes in position of objects and environ- 
ments through time? What do students know about the relative 
movements of the Earth, Moon, Sun, and the planets? The content 
to be assessed in Earth science centers on objects and features that 
are relatively accessible or visible: the solid Earth (lithosphere), 
water (hydrosphere), air (atmosphere), and the Earth in space. With 
respect to Earth science, the NAEP Science Assessment should 
focus on the following concepts and topics: 

Solid Earth: 

• Composition of the Earth 

• Forces that alter the Earth’s surface 

• Rocks: their formation, characteristics, and uses 

• Soil: its changes and uses 

• Natural resources used by humankind 

• Forces within the Earth 

Water: 

• Water cycle 

• Nature of the oceans and their effects on water and climate 

• Location, distribution, and characteristics of water, and its 
effect and influence on human activity 

Air: 

• Composition and structure of the atmosphere, including 
energy transfer 



Nature of weather 



• Common weather hazards 

• Air quality and climate 

Earth in Space: 

• Setting of the Earth in the solar system 

• Setting and evolution of the solar system in the universe 

• Tools and technology used to gather information about space 

• Apparent daily motions of the Sun, Moon, planets, and stars 

• Rotation of the Earth about its axis and the Earth’s revolution 
around the Sun 

• Tilt of the Earth’s axis that produces seasonal variations in 
climate 

• Earth as a unique member of the solar system that may be 
approximated in other galaxies in the universe, and that 
evolved at least 4.5 billion years ago 

Physical Science 

The physical science component of the 1996 and 2000 NAEP 
Science Assessments relates to basic knowledge and understanding 
concerning the structure of the universe as well as the physical 
principles that operate within it. The assessment should probe the 
following major topics: matter and its transformations, energy and 
its transformations, and the motion of things. The NAEP Science 
Assessment should focus on the following physical science concepts: 

Matter and Its Transformations: 

• Diversity of materials: the classification, types, and particulate 
nature of matter 

• Temperature and states of matter 

• Properties and uses of material: modifying properties and the 
synthesis of materials with new properties 

• Resource management 
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Energy and Its Transformations: 

• Forms of energy 

• Energy transformations in living systems, natural physical 
systems, and artificial systems constructed by humans 

• Energy sources and use, including distribution, energy conver- 
sion, and energy costs and depletion 

Motion: 

• An understanding of frames of reference 

• Force and changes in position and motion 

• Action and reaction 

• Vibrations and waves as motion 

• General wave behavior 

• Electromagnetic radiation 

• Interactions of electromagnetic radiation with matter 

Life Science 

The fundamental goal of life science is to attempt to understand 
and explain the nature and function of living things. During the 
20th century, the focus of biological research has changed from 
descriptive natural history to experimental investigation, with evol- 
ution as the central, unifying theory. The following list contains the 
major concepts to be assessed in life science: 

Change and Evolution: 

• Diversity of life on Earth 

• Genetic variation within a species 

• Theories of adaptation and natural selection 

• Changes in diversity over time 



Cells and Their Functions: 

• Information transfer 



• Energy transfer for the construction of proteins 

• Communication among cells 

Organisms: 

• Reproduction, growth, and development 

• Life cycles 

• Functions and interactions of systems within organisms 

Ecology: 

• interdependence of life: populations, communities, and 
ecosystems 

Knowing and Doing Science 

In the 1990 NAEP Science Assessment, three categories were 
used: “knowing science,” “solving problems,” and “conducting 
inquiries.” For the 1996 and 2000 NAEP Science Assessment, it 
should be noted that not only has the “knowing science” dimension 
been changed, but its meaning has been redefined as well. It has 
been reformulated as “conceptual understanding of science” to 
stress the connections and the organization of factual knowledge 
in science. 

The subdimension of “scientific investigation” has been substi- 
tuted for the 1990 category of “conducting inquiries.” This new 
subcategory is intended to probe students’ abilities to use the tools 
of science, including both cognitive and laboratory tools. Appropri- 
ate to their age and grade level, students should be able to acquire 
new information, plan appropriate investigations, use a variety of 
scientific tools, and communicate the results of their investigations. 

The 1990 category “solving problems” proved to be ambiguous. 
To emphasize the need to assess students’ abilities to use and apply 
science understanding in new, real-world applications, this cate- 
gory has been redefined as “practical reasoning.” Practical reason- 
ing subsumes competence in analyzing a problem, planning and 




20 





evaluating appropriate approaches, carrying out the required 
procedures for the approach(es) selected, and evaluating the 
result(s). 

Each of the three categories is described below. 

Conceptual Understanding 

Mastery of basic scientific concepts can best be shown by a stu- 
dent’ s ability to use information to conduct a scientific investiga- 
tion or engage in practical reasoning. Optimally, essential scientific 
concepts involve a variety of information, including: 

• Facts and events learned from science instruction and through 
experiences with the natural environment 

• Scientific concepts, principles, laws, and theories that scien- 
tists use to explain and predict observations of the natural 
world 

• Information about procedures for conducting scientific 
inquiries 

• Procedures for the application of scientific knowledge in the 
engagement of practical tasks 

• Propositions about the nature, history, and philosophy of 



• Kinds of interactions between and among science, technology, 
and society 

The goal of school science is to engender conceptual understand- 
ing. Students should acquire information in ways that will enable 
them to apply it efficiently in the design and execution of scientific 
investigations and in practical reasoning. 

A challenge in the design of assessment exercises is to capture 
changes in the characteristics of student performance as children 
mature. In the primary years, when the goal of school science is 
to build a rich collection of information derived from examined 
experiences with the natural environment, the assessment of con- 
ceptual understanding will focus on the breadth of information 
about the natural world and the student’s ability to elaborate on 
principles by using personal experiences. Does the student know 
the cyclical changes in the apparent shape of the Moon over time? 



science 
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More importantly, can the student relate how he or she knows about 
the changes? What evidence does the assessment exercise provide 
that the student’s information is based on direct experience? Is there 
a science notebook in which the student recorded observations of 
the Moon over time? Does the student know that sometimes the 
Moon is visible during daylight hours? In the primary years, the 
focus should not be on explanation or prediction, but instead on 
knowledge obtained from rich experiences in school. Consequently, 
assessment exercises would not be concerned with having students 
explain why the Moon appears to change shape but rather with 
relationships between time of day, apparent positions of the Sun 
and the Moon, and times of moonrise and sunset. 

In the middle and high school years, the emphasis should shift 
from richness of experience to reasonable scientific interpretation 
of observations. In the elementary years, the primary concern 
should be with how well reasoned an interpretation the student 
presented, not with whether it reflects the most sophisticated 
scientific reasoning. However, at grades 8 and 12, the assessment 
should be increasingly concerned with the congruence of the 
students’ interpretations' with accepted interpretations, as well as 
with the sophistication of their reasoning in moving from observa- 
tions of the natural world to explanations and predictions. Of 
special interest in the 1996 and 2000 NAEP Science Assessment 
will be the extent to which students are able to understand and use 
the notions of models, systems, and patterns of change. 

It is important to note that many aspects of conceptual under- 
standing as defined for the new NAEP Science Assessment cannot 
be tested using exclusively multiple-choice items. Items of this 
kind may be satisfactory for assessing individual parts of the infor- 
mation base, but they are limited in tapping highly valued aspects 
of conceptual understanding. 

Scientific Investigation 

Scientific investigation represents the activities of science that 
distinguish it from other ways of knowing about the world. It 
incorporates such previously used assessment categories as “pro- 
cesses of science” and “scientific problem solving.” This category 
is not just another name for the “scientific method.” Indeed, there 
is great confusion about the scientific method in the teaching of 
science. Real science is doing what one can in any way one can. 



often creatively and insightfully and using flashes of insight with 
little regard for a progression of steps. However, there is a familiar 
format and context for reporting the results of experiments. It begins 
with the report of the problem and continues with the hypothesis, 
the experimental design, the data collected, the analysis of those 
data, and the conclusions, if any. This convention of science is often 
mistaken for how scientists actually work. The results must satisfy 
logical analysis, but the logical ordering may appear only when the 
report is prepared. A great disservice has been done to generations 
of students because well-meaning people have taught the standard 
method of reporting science as the standard method of doing 
science. 

Scientific investigations must be designed at levels appropriate 
to the development of the students. This component has important 
implications for assessment. Young students are limited in their 
ability to perceive the scale of things, both very large and very 
small. Students’ limitations handicap them when they are forced, 
either by the textbook or by the curriculum, to deal with develop- 
mentally inappropriate concepts such as atoms or even cells. Young 
students are also developmentally limited in their ability to under- 
stand time. The distant past and the future are narrowly perceived 
by the egocentric student. Instruction, as well as assessment, must 
recognize where the student is and take developmental levels into 
account. As students develop and accumulate experiences, their 
performance in doing scientific investigations should begin to look 
more and more like “real” science. 

Central to the ways scientists work is the concern for a fair test, 
a controlled experiment. Children seem to have an intuitive sense 
of what makes a fair test. What they lack is the ability to consider 
all the variables and the means to control the variables. It might 
be reasonable to consider a developmental continuum such as the 
following when thinking about the control of variables: 

1. The first level of variables contains the simplest type of 
variable — the nominal variable. Nominal variables have two 
or more unordered values: “This plant was watered; that plant 
was not.” “This seed was placed in the sunlight, and that one 
was placed in the dark.” 

2. The second level of variables is the ordinal variables level. 
These variables have a sequential order and no determined 
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intervals (for example, the sequential ordering of objects by 
relative weight). 

3. The third level of variables is called the continuous variables 
level. These variables have sequence and equal intervals and 
are on a continuous scale: “This object has a temperature 

of 50 degrees Celsius, and that object has a temperature of 
57 degrees Celsius.” 

4. Tlie fourth level of variables is called the ratio variables level. 
These variables are similar to the continuous variables but 
have an absolute beginning point (for example, Kelvin temp- 
erature scale with an absolute zero point). 

As students are asked to demonstrate their ability to do scientific 
investigations, it is important to keep in mind this sort of develop- 
ment in understanding and performance, not just with respect to the 
control of variables but also regarding the other elements of doing 
science. The difficulty with the assessment may not be the content 
but the level of variable embedded in the content. 

Practical Reasoning 

Practical reasoning about matters with scientific content — the 
ability to apply one’s knowledge, thought, and action to real 
situations, not “textbook problems” — is influenced by the ability to 
(1) abstract and consider hypothetical experiences, (2) consider 
several factors simultaneously, (3) take a depersonalized view, 
and (4) realize the importance of practical reasoning and life 
experience. These factors develop throughout life. 

One of the characteristics of young children is that they have 
difficulty dealing with multiple ideas simultaneously. With matu- 
rity and experience, they can consider several ideas at once and 
weigh benefits in relation to costs or risks. The ability to abstract 
and consider hypothetical situations develops as students progress 
in science and learn to deal with more remote phenomena and 
generalizations. 

As they mature, students also learn to take depersonalized views 
of situations and to consider other people’s points of view. Often 
real-life problems involve not only theoretical and technical ele- 
ments, but also personal preferences. What will be the social impact 
of a new waste disposal system? What will neighbors say if a traffic 
light is installed? How will other students react if the lunchroom 
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noise is diminished by staggering the lunch hour? To consider these 
questions carefully, it is necessary to understand different perspec- 
tives. The ability to understand the viewpoints of others increases 
with age and experience. 

Young children also may not realize the need for scientific 
information in solving problems. For example, children below the 
age of 12 usually see no need to carry out measurements (Strang, 
1990). Also, because young children have little responsibility 
for decisions affecting their lives, they may not see the need for 
practical reasoning. However, the more that students have done 
or seen, the rnore likely it is that they can solve real-world prob- 
lems. With age and experience, the possibility increases that a new 
situation is analogous to a previous one and that the human, techni- 
cal, and theoretical factors involved in a new situation have already 
been encountered. 

All these factors suggest that practical reasoning should become 
a major factor in science assessment at grades 8 and 12 rather than 
at grade 4. As students become eager to take control of their lives, 
wish to try out their understanding of the world, and progress 
in development, practical situations related to their everyday life, 
school, and home provide excellent exemplars to demonstrate 
science-related practical reasoning. Thus, students might be asked 
to discuss problems such as noise abatement in the lunchroom, to 
design a simple apparatus such as a flashlight or a burglar alarm, or 
to plan a school garden. 

By grade 12, students should be able to discuss larger science- 
and technology-linked problems not directly related to their imme- 
diate experiences. Examples of these might be waste disposal, 
energy uses, air quality, water pollution, noise abatement, and the 
tradeoffs between the benefits and adverse consequences of various 
technologies. 

The Nature of Science 

Knowledge of the nature of science is central to the understand- 
ing of the scientific enterprise. Yet often this category is relegated 
to a discussion — or even rote memorization — of some version 
of the “scientific method.” There is total agreement that the topic 
should play a prominent role in the 1996 and 2000 NAEP Science 
Assessment. 
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A controversy existed within the project committees concerning 
whether this category is sufficient unto itself, or whether the Frame- 
work should include a separate section that deals with the nature of 
technology. The project committees were split on this issue. How- 
ever, all acknowledged that technology is integral to the nature of 
science and ought to be included in the assessment provided that it 
clearly does not exist as a separate subcategory within the assess- 
ment. Technology, then, will be measured as it relates to science 
and the scientific enterprise. 

Science 

The following concepts are appropriate for assessment at the 
given levels: 

• By grade 4, students should understand that science is trying 
to find out what happens in the natural world. Through care- 
ful observation of objects and events, they should be able to 
develop explanations for their observations. Students should 
also understand that different people may notice different 
things, and therefore may explain things differently. 

• By grade 8, students should have acquired an understanding 
of the control of variables and the difference between show- 
ing that conditions occur together and that they are causally 
related. Students should grasp what makes for a good scien- 
tific explanation by using all the relevant observations; sug- 
gesting what new observations to make; and explaining, as 
simply as possible, a wide variety of observations. 

• By grade 12, students should demonstrate their knowledge 
and understanding of the following: 

— Scientific conclusions are based on logic and evidence, 
but no fixed series of steps make up a “scientific method.” 
Scientists try to invent explanations that are logical and that 
fit observations, but these are subject to change based on 
new evidence. 

— Explanations are most believable when they also account 
for observations that were not known to the explainer. 

— Scientists (like anyone else) tend to look for, pay attention to, 
and cite evidence that supports what they already believe. 
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—New conclusions require that scientists consider all possible 
objections to their own findings. 

— Scientific organizations try to avoid bias and maintain 
quality by having scientists’ reports of observations and 
explanations judged by other scientists before they can be 
published. 

— -Few human problems can be solved with scientific knowl- 
edge alone. Most are too complicated and involve values, 
about which science has little to say. 

Technology 

Students are surrounded by and interact with the manmade world 
as much as with the natural world. Therefore, they must develop an 
understanding of what shapes the design and development of the 
technologies that are a part of that manmade world and their daily 
lives. Rather than being a content area, technology is embedded 
within this section because of its close association with science. 

The following concepts are appropriate for assessment at the given 
levels: . 

• gr^de 4, students should understand that any design requires 
making tradeoffs and that advantages and disadvantages must 
be weighed. 

• By grade 8, students should understand that scientific knowl- 
edge is often useful in design and that much scientific investi- 
gation is done for the purpose of improving design. They 
should understand that there are often several ways to solve a 
design problem and that possible solutions should be evalu- 
ated on and justified by their advantages and disadvantages. 

• grsde 12, students should know that scientific knowledge 
may help to predict consequences of one design or another, 
but that design decisions often depend upon human values 
that are outside of science. They should also be able to apply 
scientific concepts to scientific, societal, and/or technical con- 
cerns. They should understand that every design has limits 
and may fail if required to work outside of these limits. 
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Themes 



Themes are the “big ideas” of science that transcend the various 
scientific disciplines and enable students to consider problems 
with global implications. To understand the conceptual basis for 
the themes that have been selected, students must begin to develop 
an understanding of major ideas by the 4th grade. They should 
continue to develop their understanding through the 8th grade, and 
by the 12th grade they should have the ability to integrate their 
knowledge and understanding. 

The review of current state frameworks conducted in the course 
of developing the new NAEP Assessment Framework revealed that 
many are based in part on crosscutting themes in science. Several 
national organizations, including the American Association for 
the Advancement of Science, have issued reports that advocate the 
importance of common themes. The number of themes defined in 
these reports and state frameworks varies somewhat, but there is 
considerable agreement on which common elements or big ideas 
of science should be understood by students as they complete their 
high school education. The decision by the NAEP Science Assess- 
ment committees to include themes underscores their emerging 
importance, as well as the necessity to integrate themes through 
programmatic threads into grades K-12. Three of the themes are 
common to all the documents: Models, Systems, and Patterns 
of Change. These three themes were included in the 1996 NAEP 
Assessment because they constitute major, interdisciplinary organiz- 
ing principles of science. Further, they do not conflict or compete 
with the factual content of the various fields, but rather augment 
and help organize that information into a coherent intellectual 
framework. 

Models of objects and events in nature can be used to understand 
complex or abstract phenomena. Models may be first attempts to 
help identify the relevant variables to build evermore useful repre- 
sentations, or they may be highly refined for predictions about the 
actual phenomenon. Students need to understand the limitations 
and simplifying assumptions that underlie the many models used 
in the natural sciences. A model is likely to fit data well only within 
a limited range of circumstances and to be misleading outside of 
that range. 



Systems are complete, predictable cycles, structures, or pro- 
cesses occurring in natural phenomena, but students should under- 
stand that the idea of a system is an artificial construction created 
by people for certain purposes — to gain a better understanding of 
the natural world or to design an effective technology. The con- 
struct of a system entails identifying and defining its boundaries, 
identifying its component parts and the interrelations and intercon- 
nections among those parts, and identifying the inputs and outputs 
of the system. 

Regardless of the topic around which the Patterns of Change 
theme is developed, students should be able to recognize patterns 
of similarity and difference, to recognize how these patterns change 
over time, to remember common types of patterns, and to transfer 
their understanding of a familiar pattern of change to a new and 
unfamiliar situation. Appendix C contains more detail on these 
three themes and the developmentally appropriate expectations for 
students at grades 4, 8, and 12. 



Chapter Three 

Desired Attributes of the Assessment 



Some Assessment Issues 

N AEP functions to provide information about (1) the knowl- 
edge and scientific understanding of the nation’s youth and 
(2) the features of science education programs that relate to 
high levels of student achievements. Consequently, the design plan 
for the NAEP Science Assessment includes strategies for measure- 
ment in both of these areas. Research has produced practical and 
theoretical knowledge that is important to the assessment design 
process. 

The Nature of Testing and the Nature of 
Knowledge and Learning 

Except for early assessments, NAEP science tests have consisted 
primarily of short-answer, paper-and-pencil questions that were 
mostly multiple choice. The previous assessments tended to focus 
on discrete components of science, each of which was usually 
learned independently of the others. Hence, tests were made up of 
independent items, each comparable to the others and weighted the 
same. 

The Framework is based on a different view. It holds that scien- 
tific knowledge should be organized to provide a structure that 
connects and creates meaning for factual information, and that this 
organization is influenced by the context in which the knowledge is 
presented. Learning is perceived as an activity in which the learner 
interacts with the physical world, with peers and teachers, and with 
the scientific community. In this view, science proficiency depends 
upon the ability to know facts and integrate them into larger con- 
structs and to use the tools, procedures, and reasoning processes of 
science for an increased understanding of the natural world. 

Rather than concentrate on facts in isolation, assessments will 
reflect the organization and structure of scientific knowledge and 



the nature of learning in science. Because scientific knowledge is 
expanding faster than can be accommodated by any curriculum, 
teachers and assessment designers must make choices about what 
topics, concepts, and factual information to address. Consequently, 
the 1996 and 2000 NAEP Science Assessment Framework con- 
centrates on assessing students’ ability to relate basic facts and 
concepts as well as their ability to discuss and evaluate approaches 
to science-related problems. The Framework also stresses that 
an assessment of what students know and can do must employ 
techniques reflective of the nature of science. 

Inferring Understanding From Student Responses 

Test items present students with tasks that may require a range 
of responses, from recall of factual information to performance of 
scientific investigations to complex reasoning. Based on analysis of 
the responses, experts in the field make inferences about students’ 
understanding — that is, the knowledge and reasoning skills that are 
assumed to have produced the responses. The validity of these 
inferences is a central issue in assessment. 

Responses to exercises designed to assess thinking or mental 
processing are generally more difficult to intejpret than responses 
to items designed to assess factual knowledge. In practice, basing 
an assessment of the quality of mental processing on short respon- 
ses is problematic. Often the decision about the mental processes 
applied is based only on the accuracy of the factual knowledge 
in the answer. When the answer is factually correct, the observer 
infers that the mental processes represent scientific reasoning (for 
example, those mental processes that are necessary to understand 
information in the test question stem, to retrieve scientific knowl- 
edge from memory, to reason from the stem to the correct response, 
or to eliminate incorrect responses). But this is not necessarily a 
valid inference. An incorrect answer may be the result of misinfor- 
mation^ not flawed reasoning; a correct answer is not necessarily 
the product of sound reasoning. Illogical thinking or using a wrong 
assumption or incorrect information can produce a seemingly 
correct answer. Furthermore, a correct answer may not require any 
higher order thinking at all; it simply may have been recalled. Only 
if the student response includes some indication of how the answer 
was obtained will those who score the assessment have information 
from which to choose among alternative interpretations. 
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Science entails observing objects and phenomena in the natural 
world and collecting and interpreting information about them. 

For this reason, pencil-and-paper tests have been criticized as too 
limited for assessing what students know and can do in science. 

Over the past several years, groups have developed assessment 
exercises that engage students in “performance tasks” using scien- 
tific equipment and materials. Student responses are recorded by an 
observer or by the students themselves in written form. However, 
these exercises have limitations as well. 

Indeed, in the course of developing the new Framework, numer- 
ous examples of perform^ce exercises were examined with respect 
to science concepts and reasoning, but many did not stand up to 
rigorous analysis. These exercises might have led science teachers 
and educators to draw faulty inferences. 

The following is typical of a performance exercise that is coun- 
terproductive: An exercise requires a student to identify several 
unknown substances by means of indicators, but the student is given 
minutely detailed directions for performing each step in the identifi- 
cation process. Unfortunately, even if the answers are correct, the 
only inference to be drawn is that the student can follow written 
instructions. A test item fonhulated with such detailed step-by-step 
directions reduces to zero the science understanding needed for 
problem solving. 

Many of the so-called performance assessment tasks that were 
reviewed turned out to be standard laboratory exercises that, again, 
were reduced to “follow-the-instructions” problems. No inferences 
about a student’s knowledge of science or its tools and procedures 
can be drawn from such exercises. To test higher order thinking 
skills — a major goal of performance assessments — problems need 
to be placed in new contexts, applied to new situations, or have 
new elements introduced that preclude students’ recalling what they 
have done before (Resnick, 1987). 

Class, culture, ethnicity, gender, language ability, and access to 
quality instruction may influence the manner in which science is 
learned and the manner in which science attitudes and knowledge 
are produced. Hence, individuals need opportunities to demonstrate 
knowledge or competencies in different contexts. 

Assessment techniques that show group differences are more 
likely to reveal problems with student learning and classroom 



instruction than with assessment, per se. However, this does not 
eliminate the assessment community’s responsibility to the broader 
society. In addressing the issues of pluralism, multiple assessment 
i^^thods niay be more effective than any one method — no matter 
how well it is developed. 

Developments in Assessment 

Currently, some state pilot-testing efforts are providing new 
ideas about assessment exercise and task formats. These pilot 
activities are also aimed at assessing new types of information 
relevant to new curriculum guides emerging in the states. 

The experimental assessment work (much of it pioneered in the 
United Kingdom) uses new approaches, including performance 
tasks, open-ended tasks, and new types of multiple-choice questions 
that are thematic or conceptual or that ask students to explain their 
choices in short written answers. Moreover, state assessments are 
experimenting with collecting information on other meaningful 
outcomes: sustained student work, proficiency in designing and 
conducting experiments, and fluency of ideas critical to the natural 
sciences and related fields. They are also experimenting with inno- 
vative reporting approaches. The performance measures are created 
through consensual judgment about what students should know and 
be able to do at given grade levels or developmental stages. 

Criteria for Assessing Learning and 
Achievement in Science 

The Framework for the 1996 and 2000 NAEP Science Assess- 
ments has been developed according to the following broad 
guidelines: 

• By focusing on meaningful knowledge and skills, NAEP 
should be a force in fostering progress as well as measuring it, 
enabling more students to learn more science. 

• A range of assessment means must be used, including some in 
which the student is required to create and construct, not just 
to recognize and respond. 

• Assessment exercises should challenge students at develop- 
mentally appropriate levels to: 



— Explain commonplace natural phenomena using appropriate 
scientific theory, principles, and concepts. 

— Plan the investigation of a novel scientific problem. 

—Demonstrate understanding of the basic knowledge struc- 
tures of science by using the appropriate techniques to 
connect concepts to one another and to the theory within 
which they are embedded. 

— Demonstrate some understanding of pervasive crosscutting 
themes in science. 

— Solve practical problems by using the appropriate theories, 
principles, concepts, and techniques of science. 

• Assessments must be sensitive to the need and ability of 
students to function in a variety of contexts. 

• Assessment exercises should use a variety of formats to allow 
students to display the wide range of competencies expected 
as the outcomes of science education. 

• Assessment tasks that are larger than single items should be 
analyzed in multiple ways, not restricted to providing inform 
mation for single scales. 

• Test results should not be normalized — that is, students’ out- 
comes should not be manipulated to fit a normal distribution 
curve. 

• Assessments must have enough questions about enough topics 
to explore students’ knowledge in depth. 

Achievement Levels in Science 

Achievement levels describe how well students should perform 
on the content and thinking levels required by the assessment. They 
evaluate the quality of the outcomes of students’ education in 
science at grades 4, 8, and 12 as measured by NAEP. 

Three achievement levels — ^Basic, Proficient, and Advanced — 
have been defined for each grade level assessed by the National 
Assessment Governing Board. 

Basic denotes partial mastery of prerequisite knowledge and 
skills that, are fundamental for proficient work at each grade. 



Proficient represents solid academic performance for each grade 
assessed. Students reaching this level have demonstrated compe- 
tency over challenging subject matter, including subject-matter 
knowledge, application of such knowledge to real-world situations, 
and analytical skills appropriate to the subject matter. Advanced 
represents superior performance. 

Appendix A lists the final achievement level descriptions for 
students participating in the 1996 and 2000 NAEP Science Assess- 
ment in grades 4, 8, and 12. The assessment was constructed to 
measure and report student performance according to the three 
levels of achievement, as required by NAEP policy. 
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Chapter Four 



Characteristics of Assessment 
Exercises 

Types of Exercises 

I nnovative assessments in the United States and other countries 
use three major item types: performance exercises, open-ended 
paper-and-pencil exercises, and multiple-choice items probing 
understanding of conceptual and reasoning skills. A fourth type 
often added to time-limited tests is the collection and evaluation of 
portfolios of student work done in the course of instruction. There 
is also an emerging assessment technique that involves two-phase 
testing. The following,sections discuss these exercise types and 
provide some guidelines for the amount of assessment time to be 
devoted to each. Further details are provided in the Specifications 
and Reporting Formats and Issues documents. 

Performance Exercises 

In performance exercises, students actually manipulate selected 
physical objects and try to solve a scientific problem about the 
objects before them. Although various types of performance tests 
have been piloted extensively, their standardization and administra- 
tion differ widely. One method for ensuring uniform administration 
is the use of standardized performance test kits, with each test proc- 
tored and scored by trained personnel. Depending on the objectives 
established for the assessment, student answer sheets can also be 
used to provide responses to be scored. To adequately measure 
the goals outlined for the assessment, performance items generally 
should make up at least 30 percent of the assessment, as measured 
by student response time. An extra period of time (20 or 30 min- 
utes) may be necessary for students who have been assigned to 
perform complex tasks. 

The shortcomings of many performance tasks currently being used 
were discussed in the preceding chapter. How can a performance 
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exercise be designed so thdt it meets criteria for assessing science 
concepts and their relations? The exercise should be meaningful and 
not a context-free laboratory problem. Personal context, for example, 
is seen in the following problem: “You have just been given this 
new drink. It is claimed to be sugar- and calorie-free. What can you 
find out about the accuracy of these claims with these indicators?” 
Students can be given the names and procedures for the safe use of 
each indicator, but no information about their scientific use. The 
students would have to know, for example, that iodine solution is a 
test for starch and what the negative and positive reactions are. The 
students would also have to know that if fats, proteins, or carbohy- 
drates are present, they will yield calories. They would have to plan . 
how to conduct an investigation of the unknown in such a way as 
not to waste the materials, to be able to repeat the investigation when 
they believe their procedure is faulty, and to have enough solution 
left to replicate the investigation for verification. They would also 
have to design their data-collecting procedures. Finally, they would 
need to interpret and justify their findings. The questions asked of 
students as part of a performance exercise need to enable the students 
to display understanding and to justify interpretations. Such ques- 
tions as “What substance is in the unsown?” or “How far did the 
dye diffuse?” do not elicit responses that demonstrate understanding. 

If students need additional information to carry out the task, 
they could be asked before they begin if they would like any other 
materials and why. If they request known substances for each 
indicator to refresh their memories, those could be provided. Such 
a request demonstrates one aspect of understanding the processes 
of science — ^knowing what one doesn’t know and how to acquire 
more information. Scoring for such a problem could give points 
for science knowledge, for laboratory procedures, and (if using an 
observer) for a systematic approach to problem solving in contrast 
to a trial- and-error or random approach. 

Open-Ended Paper-and-Pencil Items 

Open-ended items that require written responses provide partic- 
ularly useful insights into students’ levels of conceptual under- 
standing. They can also be used to assess students’ abilities to 
communicate in the sciences. In addition, open-ended items, if 



carefully crafted, can be used to reflect students’ abilities to gener- 
ate rather than recognize information related to scientific concepts 
and their interconnections. Open-ended items should liiake up at 
least 50 percent of the assessment, as measured by student response 
time. About one-third of the open-ended questions should consist of 
extended response items. 

Multiple-Choice Items 

The 1996 and 2000 NAEP Science Assessment will send impor- , 
tant messages about science curriculum and classroom instruction. 
The use of multiple-choice items should be considered carefully 
because they are often overused to test low-level fecall. Balanced 
with other item types, however, multiple-choice items are worth- 
while for measuring knowledge of important facts and concepts as 
well as deductive reasoning skills. Multiple-choice items should 
make up no more than 50 percent of the assessment, as measured 
by student response time. 

Additional Considerations 

Performance exercises, open-ended paper-and-pencil items, and 
multiple-choice items could produce responses less subject to faulty 
interpretation if students are given an opportunity to explain their 
responses, their reasoning processes, or their approach to a prob- 
lem, Hence, an assessment should afford this opportunity. But care 
must be taken, particularly with fourth graders, that language ability 
is not confounded with science ability. This caution also applies to 
the more complex multiple-choice items needed to probe concep- 
tual understanding. 

The new NAEP Science Assessment, to be consonant with current 
reform efforts in science education, needs to probe students’ depth of 
knowledge and scientific understanding. For this reason, it is recom- 
mended that for at least half the students sampled, the assessment 
include an indepth examination involving a single problem or topic. 
The format could be a set of linked performance tasks, open-ended 
paper-and-pencil exercises, multiple-choice items, or a combination 
thereof. Pending modification after pilot testing, the suggested time 
to be spent by students on this type of exercise is 10 minutes for 
grade 4; 20 minutes for grade 8; and 30 minutes for grade 12. 
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Pilot Testing 

Multiple approaches need to be tried in the pilot testing of the 
assessment exercises. It would be especially useful to test the same 
concept(s) and performance skill(s) in different ways to see which 
method provides the richest, most reliable, and valid information. 
For example, if an open-ended question can easily be turned into a 
multiple-choice question without losing its intent and validity, it 
should be multiple choice. Open-ended questions should tap skills 
and knowledge that are truly “open”— probing for the integrated 
application of relevant knowledge, not for the recall of a series of 
unconnected facts. The following additional issues need to be 
investigated during pilot testing: 

• Scoring rubrics. These should be developed a priori for 
open-ended questions and performance tasks, but modified 
on the basis of pilot test results. Weights should be assigned 
within scoring rubrics of complex items to reflect the quality 
of the responses. Distinct rubrics must be developed to score 
for multiple aspects or for items contributing to more than one 
component of the assessment. 

• Scaffolding. To what extent should scaffolding or providing 
additional information be integrated into open-ended ques- 
tions? This could be done either in the form of “hints” (to be 
given only after students get stuck, for example, in a computer- 
administered or individually timed exercise) or by informing 
the students how their answers will be scored. How much 
should students be able to learn during the test, as contrasted 

to what they learned in the classroom? Unfortunately, many 
science items in current tests require no science knowledge at 
all because everything is provided in the question stem or in 
the instructions. On paper-and-pencil tests, these questions 
usually turn out to measure general reasoning or scale and 
graph reading. 

• Student self-evaluation. Should students (perhaps in grades 

8 and 12 only) be able to evaluate their own test performance? 
Students might be given credit for what they know they don’t 
know, particularly if they can articulate what steps they might 
take to find out the answer. Alternatively, they might be asked 
to indicate how well they think they did on some items 
(metacognitive questions). 
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Procedures for the Development of the Assessment 

The planning committee responsible for developing the new 
NAEP Science Assessment Framework was concerned that the 
nature and specifics of the Framework be faithfully mirrored in 
the actual instrument of the assessment. The committee therefore 
recommended that a detailed review of individual items and of each 
proposed assessment as a whole be conducted at the conclusion of 
the following four stages: 

• Item development and selection for the pilot testing. 

• Analysis of data from the pilot test arid review of the results, 

• A priori development of the scales at each grade level, to- 
gether with review of the items used for the behavioral 
anchoring. 

• Selection of items and formulation of the assessment as a 
whole for the NAEP test administration. 

The review committee(s) should be broadly constituted to 
include scientists; teachers; science educators and researchers; 
cognitive psychologists; psychometricians; and informed members 
of the lay community, including representatives from business, 
higher education, education governing bodies, and parent groups. 
The stages of review should be carried out by the same committee. 
If this is not feasible, the different committees need to have over- 
lapping membership. Members of the planning committee should 
be included throughout the entire review process. 

Special Study in Science 

Special studies are often recommended as a part of the National 
Assessment process because new or emerging techniques offer 
promise and, if they yield useful information, will make a positive 
contribution to the assessment process. Special studies are a part of 
the main NAEP process and are usually reported along with the 
results from the national sample. 

In 1996, a special study was carried out to assess students with 
advanced training in science. Past NAEP science assessments have 
been criticized for having too low a ceiling — that is, not including 
an adequate number of items at advanced levels of difficulty. As a 
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result, NAEP tests are assumed not to have reflected what the best 
prepared students know or can do in science. This issue has become 
more serious since the formulation of the National Education Goals 
particularly goal 4: “By the year 2000, U.S. students will be first in ’ 
the world in science and mathematics achievement.” An interna- 
tional assessment conducted in 1995 included students who are 
concentrating on science in high school. It would be highly desir- 
able to have similar results from NAEP so that reports on progress 

toward the National Education Goals will be based on more than 
one study. 

Suggestions on how to include the most competent students 
in this special study included sampling students who are taking 
advanced courses in science. Assessments must contain a sufficient 
number of challenging exercises to measure what these “best” 
students know and are able to do. It was recommended, therefore, 
that a special study be conducted in 1996 with a subsample of the 
national NAEP sample to determine whether this is a useful ap- 
proach to establish the achievement and performance of the best 
science students. 
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1996 and 2000 NAEP Science 
Achievement Level Descriptions 



Grade 4 
Basic 

Students performing at the Basic level demonstrate some of the 
knowledge and reasoning required for understanding of the Earth, 
physical, and hfe sciences at a level appropriate to grade 4. For ex- 
ample, they can carry out simple investigations and read uncomph- 
cated graphs and diagrains. Students at this level also show a beginning 
understanding of classification, simple relationships, and energy. 

Fourth-grade students performing at the Basic level are able to follow 
simple procedures, manipulate simple materials, make observations, 
and record data. They are able to read simple graphs and diagrams 
and draw reasonable but limited conclusions based on data provided 
to them. These students can recognize appropriate experimental 
designs, although they are unable to justify their decisions. 

When presented with diagrams, students at this level can identify 
seasons; distinguish between day and night; and place the position 
of the Earth, sun, and planets. They are able to recognize major 
energy sources and simple energy changes. In addition, they show 
an understanding of the relationship between sound and vibrations. 
These students are able to identify organisms by physical character- 
istics and group organisms with similar physical features. They can 
also describe simple relationships among structure, function, 
habitat, life cycles, and different organisms. 

Proficient 

Students performing at the Proficient level demonstrate the knowl- 
edge and reasoning required for understanding of the Earth, physi- 
cal, and life sciences at a level appropriate to grade 4. For example, 
they understand concepts relating to the Earth’s features, physical 
properties, structure, and function. In addition, students can formu- 
late solutions to familiar problems as well as show a beginning 
awareness of issues associated with technology. 

Fourth-grade students performing at the Proficient level are able to 
provide an explanation of day and night when given a diagram. 
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They can recognize major features of the Earth's surface and the 
impact of natural forces. They are also able to recognize water in its 
various forms in the water cycle and can suggest ways to conserve 
it. These students recognize that various materials possess different 
properties that make them useful. Students at this level are able to 
explain how structure and function help living things survive. They 
have a beginning awareness of the benefits and challenges associ- 
ated with technology and recognize some human effects on the 
environment. They can also make straightforward predictions and 
justify their position. 

Advanced 

Students peifomiing at the Advanced level demonstrate a solid 
understanding of the Earth, physical, and life sciences as well as the 
ability to apply their understanding to practical situations at a level 
appropriate to grade 4. For example, they can perform and critique 
simple investigations, make connections from one or more of the 
sciences to predict or conclude, and apply fundamental concepts to 
practical applications. 

Fourth-grade students performing at the Advanced level are able to 
combine information, data, and knowledge from one or more of the 
sciences to reach a conclusion or to make a valid prediction. They can 
also recognize, design, and explain simple experimental procedures. 

Students at this level recognize nonrenewable sources of energy. 
They also recognize that light and sound travel at different speeds. 
These students understand some principles of ecology and are able 
to compare and contrast life cycles of various common organisms; 

In addition, they have a developmental awareness of the benefits 
and challenges associated with technology. 

Grade 8 
Basic 

Students performing at the Basic level demonstrate some of the 
knowledge and reasoning required for understanding of the Earth, 
physical, and life sciences at a level appropriate to grade 8. For 
example, they can carry out investigations and obtain information 
from graphs, diagrams, and tables. In addition, they demonstrate 
some understanding of concepts relating to the solar system and 
relative motion. Students at this level also have a beginning under- 
standing of cause-and-effect relationships. 
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Eighth-grade students performing at the Basic level are able to ob- 
serve, measure, collect, record, and compute data from investiga- 
tions. They can read simple graphs and tables and are able to make 
simple data comparisons. These students are able to follow direc- 
tions and use basic science equipment to perform simple experi- 
ments. In addition, they have an emerging ability to design experi- 
ments. 

Students at this level have some awareness of causal relationships. 
They recognize the position of planets and their movement around 
the sun and know basic weather-related phenomena. These students 
can explain changes in position and motion such as the movement 
of a truck in relation to that of a car. They also have an emerging 
understanding of the interrelationships among plants, animals, and 
the environment. 

Proficient 

Students performing at the Proficient level demonstrate much of the 
knowledge and many of the reasoning abilities essential for under- 
standing of the Earth, physical, and life sciences at a level appropri- 
ate to grade 8. For exarhple, students can interpret graphic informa- 
tion, design simple investigations, and explain such scientific 
concepts as energy transfer. Students at this level also show an 
awareness of environmental issues, especially those addressing 
energy and pollution. 

Eighth-grade students performing at the Proficient level are able to 
create, interpret, and make predictions from charts, diagrams, and 
graphs based on information provided to them or from their own 
investigations. They have the ability to design an experiment and 
have an emerging understanding of variables and controls. These 
students are able to read and interpret geographic and topographic 
maps. In addition, they have an emerging ability to use and under- 
stand models, can partially formulate explanations of their under- 
standing of scientific phenomena, and can design plans to solve 
problems. 

Students at this level can begin to identify forms of energy and 
describe the role of energy transformations in living and nonliving 
systems. They have knowledge of organization, gravity, and motion 
within the solar system and can identify some factors that shape the 
surface of the Earth. These students have some understanding of 
properties of materials and have an emerging understanding of the 
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particulate nature of matter, especially the effect of temperature on 
states of matter. They also know that light and sound travel at 
different speeds and can apply their knowledge of force, speed, and 
motion. These students demonstrate a developmental understanding 
of the flow of energy from the sun through living systems, espe- 
cially plants. They know that organisms reproduce and that charac- 
teristics are inherited from previous generations. These students 
also understand that organisms are made up of cells and that cells 
have subcomponents with different functions. In addition, they are 
able to develop their own classification system based on physical 
characteristics. These students can list some effects of air and water 
pollution as well as demonstrate knowledge of the advantages and 
disadvantages of different energy sources in terms of how they 
affect the environment and the economy. 

Advanced 

Students performing at the Advanced level demonstrate a solid 
understanding of the Earth, physical, and life sciences as well as the 
abilities required to apply their understanding in practical situations 
at a level appropriate to.grade 8, For example, students can perform 
and critique the design of investigations, relate scientific concepts 
to each other, explain their reasoning, and discuss the impact of 
human activities on the environment. 

Eighth-grade students performing at the Advanced level are able to 
provide an explanation for scientific results. They have a modest 
understanding of scale and are able to design a controlled experi- 
ment. These students have an understanding of models as represen- 
tations of natural systems and can describe energy transfer in living 
and nonliving systems. 

Students at this level are able to understand that present physical 
clues, including fossils and geological formations, are indications 
that the Earth has not always been the same and that the present 
is a key to understanding the past. They have a solid knowledge 
of forces and motions within the solar system and an emerging 
understanding of atmospheric pressure. These students can recog- 
nize a wide range of physical and chemical properties of matter 
and some of their interactions and understand some of the proper- 
ties of light and sound. Also, they can infer relationships between 
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structure and function. These students know the differences be- 
tween plant and animal cells and can apply their knowledge of food 
as a source of energy to a practical situation. In addition, they are 
able to explain the impact of human activities on the environment 
and the economy . 

Grade 12 
Basic 

Students performing at the Basic level demonstrate some knowl- 
edge and certain reasoning abilities required for understanding of 
the Earth, physical, and life sciences at a level appropriate to grade 
12. In addition, they demonstrate knowledge of the themes of 
science (models, systems, and patterns of change) required for 
understanding the most basic relationships among the Earth, 
physical, and life sciences. They are able to conduct investigations, 
critique the design of investigations, and demonstrate a rudiment^ 
understanding of scientific principles. 

Twelfth-grade students performing at the Basic level are able to 
select and use appropriate simple laboratory equipment and write 
down simple procedures that others can follow. They also have a 
developmental ability to design complex experiments. These 
students are able to make classifications based on definitions such 
as physical properties and characteristics. 

Students at this level demonstrate a rudimentary understanding of 
basic models and can identify some parts of physical and biological 
systems. They are also able to identify some patterns in nature and 
rates of change over time. These students have the ability to iden- 
tify basic scientific facts and terminology and have a rudimentary 
understanding of the scientific principles underlying such phenom- 
ena as volcanic activity, disease transmission, and energy transfor- 
mation. In addition, they have some familiarity with the application 
of technology. 

Proficient 

Students performing at the Proficient level demonstrate the knowl- 
edge and reasoning abilities required for understanding of the Earth, 
physical, and life sciences at a level appropriate to grade 12. In addi- 
tion, they demonstrate knowledge of the themes of science (models, 
systems, and patterns of change) required for understanding how 



these themes illustrate essential relationships among the Earth, 
physical, and life sciences. They are able to analyze data and apply 
scientific principles to everyday situations. 

Twelfth-grade students performing at the Proficient level are able 
to demonstrate a working ability to design and conduct scientific 
investigations. They are able to analyze data in various forms and 
utilize information to provide explanations and to draw reasonable 
conclusions. 

Students at this level have a developmental understanding of both 
physical and conceptual models and are able to compare various 
models. They recognize some inputs and outputs, causes and 

effects, and interactions of a system. In addition, they can correlate 
structure to function for the parts of a system that they can identify. 
These students also recognize that rate of change depends on initial 
conditions and other factors. They are able to apply scientific 
concepts and principles to practical applications and solutions for 
problems in the real world and show a developmental understand- 
ing of technology, its uses, and its applications. 

Advanced 

Students performing at the Advanced level demonstrate the knowl- 
edge and reasoning abilities required for a solid understanding of 
the Earth, physical, and life sciences at a level appropriate to grade 
12. In addition, they demonstrate knowledge of the themes of 
science (models, systems, and patterns of change) required for 
integrating knowledge of scientific principles from the Earth, 
physical, and life sciences. Students can design investigations that 
answer questions about real-world situations and use their reason- 
ing abilities to make predictions. 

Twelfh-grade students performing at the Advanced level are able 
to design scientific investigations to solve complex, real-world 
situations. They can integrate, interpolate, and extrapolate informa- 
tion embedded in data to draw well-formulated explanations and 
conclusions. They are also able to use complex reasoning skills to 
apply scientific knowledge to make predictions based on condi- 
tions, variables, and interactions. 

Students at this level recognize the inherent strengths and limita- 
tions of models and can revise models based on additional informa- 
tion. They are able to recognize cause-and-effect relationships 







within systems and can utilize this knowledge to make reasonable 
predictions of future events. These Students are able to recognize 
that patterns can be constant, exponential, or irregular and can 
apply this recognition to make predictions. They can also design a 
technological solution for a given problem. 
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Fields of Science 



The detailing of the fields of science below indicates themes that 
can be explored within major topic areas as well as brief examples 
of developmentally appropriate sample questions that can form the 
basis for assessment exercises. 

Earth Science 

Earth science is the study of the planet Earth’s composition, 
processes, environments, and history, focusing on the solid Earth 
(lithosphere) and its interactions with air (atmosphere) and water 
(hydrosphere). The content to be assessed in Earth science centers 
on objects (including bodies and materials and their composition, 
features, and structures) as well as processes and events that are 
relatively accessible or visible. The content includes objects such as 
soil, minerals, rocks and rock outcrops, fossils, rain, clouds, the 
Sun, and the Moon; processes such as erosion, deposition, weather, 
and climate; and events like volcanic eruptions, earthquakes, and 
storms. 

The Solar System (subset) 

Earth/M oon/Sun 

• ObscrvablQ evidence (Patterns of Change) 

• Description and models (Systems, Models) 

Students should understand that the apparent daily motions of 
the Sun, Moon, planets, and stars are due to the rotation of the Earth 
about its axis every 24 hours. This rotation produces the Earth’s 
night-and-day cycle. They should also understand that the Earth’s 
1-year revolution around the Sun changes how sunlight falls on one 
part or another of the Earth because of the tilt of the Earth’s axis, 
thus producing seasonal variations in climate. They should know 
that the combination of the Earth’s motion and the Moon’s own 
orbit around the Earth, once in about 28 days, results in the phases 
of the Moon. 
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The Earth and Forces That Shape It 

With respect to the Earth, the NAEP Science Assessment should 
center on the following concepts: 

Clima.tQ (Patterns of Change) 

W ater Cycle and Ground W ater ( Systems ) 

Pollution Capacity (Systems, Models) 

Interior Effects (Models, Systems, Patterns of Change) 

• Crustal plates 

• Rock cycles and strata 

Exterior Effects (Systems, Patterns of Change) 

• Weathering 

• Plants, animals, and civilization 

Students should understand that the Earth is a unique member 
of this solar system but may be replicated in other galaxies in the 
universe; that it is at least 4.5 billion years old and is a complex 
planet with several interacting systems; and that these systems have 
evolved through time and changes in them occur over periods that 
range from microseconds to millions of years and vary from sub- 
atomic to astronomical. Students should also understand that the 
Earth’s systems contain a variety of renewable and nonrenewable 
resources that sustain life (American Geological Institute, 1991). 

Climate: Atmosphere and Hydrosphere 

The atmosphere is the gaseous envelope that surrounds the 
Earth. It is continuously in motion, circulating in complex but 
regular patterns and driven by direct and stored solar energy. There 
are strong interactions between the atmosphere and the hydrosphere 
that determine weather and climate, profoundly influencing human 
and all other life. Desired learning goals: 

Grade 4 

• Students are able to communicate what is special about air. 
What do their senses tell them about the air? What needs air? 

















• Students will offer simple explanations for how the weather 
changes. How do people know when weather changes? How 
can changing weather conditions be measured? 

Grade 8 

• Students know about basic weather-related phenomena 
(for example, tornadoes, hurricanes, drought, and acid 
precipitation). 

• Students know that relatively small changes in global tem- 
peratures can have dramatic effects on the Earth. 

• Students can access climatological information bases via 
computers and other means to extract useful information. 

• Students can read a weather chart and can extract basic 
information. 

• Students understand and can use simple instruments that measure 
basic phenoniena related to weather (for example, barometers — 
students will make one and/or know how it works). 

Grade 12 

• Generally, 12th-grade students are able to connect relation- 
ships between atmospheric phenomena and long-term effects. 

• They understand that much of what determines the details of 
the weather depends on phenomena such as sea breezes, 
thunderstorms, tornadoes, and wind shear. 

• Further, they understand how scientists can monitor atmos- 
pheric events over time (for example, how products of pollu- 
tion and carbon dioxide affect the poles). 

• Students are able to discuss and relate causes and possible 
solutions to their consequences and tradeoffs. 

% 

Water Cycle and Ground Water 

Desired learning goals: 

Grade 4 

• Students understand that water exists not only on the surface 
of the Earth, but beneath it as well. 
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• Students understand that water is able to change the shape of 
the Earth. They can demonstrate their knowledge in two ways: 
by interpreting common local land features such as rivers, 
streams, mountain slopes, and delta deposits, and by designing 
simple models that illustrate their interpretations. 

• Students understand that most of the Earth’s surface is covered 
by water. 

• Students know about how waters of the Earth circulate. They can 
answer questions concerning where water is found; how water enters 
and leaves the atmosphere; and how water can be used more wisely. 

Grade 8 

• Students should know and demonstrate why water is special. 
What properties make water special? Where is water found — 
in the air, on Earth, and under the ground? 

• Students know that the oceans provide habitats for a wide 
variety of plant life and animal life. 

• Students can discuss some common problems that concern 
water — for example, its availability in their areas, shortages, 
relationship to supply and demand, and the effects of over- 
population on the availability and quality of potable water. 

• Students are able to understand common interactive cycles 
such as the water cycle, nitrogen cycle, and carbon cycle. 

• Students are familiar with some of the ways scientists explore 
the water environment. 

Grade 12 

• At grade 12, students are able to explain how water (or the 
lack of it) relates to their immediate state, city, or area. 

• They can discuss national issues related to water, such as acid 
precipitation and the effects of global warming. 

• They are able to understand how common cycles affect the 
climate of the area where they live. 

• Students should know how to find answers to problems relating 
to how laws affect the use of water as well as what hazards are 
associated with water and how they can be mitigated. 



Interaction of Earth’s Systems 

Desired learning goals: 

Grade 4 

• Students understand how the Earth relates to the Sun (for 
example, periodicity, seasons, and the night- and-day cycle). 

• Students know about tides with respect to the Earth, Moon, 
and Sun. 

• Students understand basic facts about volcanoes and glaciers. 

• Students understand how rocks and minerals can be investi- 
gated, what they are made of, and how they form. 

Grades 

• Students are able to understand how earthquake occurrences 
are recorded and note some positional regularities. 

• Students are able to connect short-term changes in climate 
with volcanic activity. 

• Students can discuss changes that have occurred in water 
levels, global temperature, and climate zones over the eons. 
They can advance some hypotheses as to why these changes 
have occurred. 

• Students can identify how and where energy is obtained from 
the Earth; Earth materials that people use and where to find 
them; and the advantages and disadvantages of using the 
Earth’ s resources. 

Grade 12 

• Students in grade 12 can connect “the zone of fire” with plate 
tectonics. They understand how sliding plates cause sudden 
Earth movements. 

• They can discuss issues related to Earth systems (for example, 
why continents move, that they have not always been arranged 
the way they are now, and how human activity affects Earth 
systems). 
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• Students can access data from more than one data source to 
develop an environmental impact statement for their region/ 
town/block. 

• Students can discuss problems associated with agriculture and 
the lithosphere. They can relate these problems to changing 
atmospheric conditions. 

Physical Science 

The physical science component of the 1996 and 2000 NAEP 
Science Assessment should probe the following major topics: 
matter and its transformations, energy and its transformations, and 
the motion of things. Each topic is detailed below, together with 
some examples of topics appropriate for assessment at the different 
grade levels. 

Matter and Its Transformations 

With respect to matter and its transformations, the 1996 and 
2000 NAEP Science Assessment should center on the following 
concepts: 

“Many From Few” 

• Diversity of materials 

• Classifications and types of materials (Patterns of Change) 

• Particulate nature of matter (Models ) 

Temperature and States of Matter 

• Uses of Materials 

— Properties and uses 

— Modifying properties by mixing, processing, and reacting 
(Patterns of Change, Models) 

— Synthesis of materials with new properties (Patterns of 
Change, Models) 

Resource Management 

• Resource depletion and substitute materials (Models) 

• Disposal and recycling (Systems) 



Materials encountered in the physical world differ greatly in 
shape, density, flexibility, texture, toughness, and color; in their 
ability to give off, absorb, bend, or reflect light; in the form they 
take at different temperatures; and in many other ways. Students 
need to understand that these varied substances are made up of 
relatively few kinds of basic materials — the atomic elements— 
combined in various ways. Only a few of these elements are 
abundant in the universe. 

As they advance in science, students should come to understand 
that the basic premise of the modem theory of matter is that materials 
consist of a limited number of different kinds of atoms (elements) 
that join together in different configurations to form substances. 

Thus, when substances react to form new substances, the elements 
composing them combine in new ways and the properties of the 
substances created by the new combinations may be very different 
from those of the old. Almost every substance can exist in a variety 
of states — solid, liquid, and gaseous — depending on temperature and 
pressure; 

Patterns within the stmcture of matter have been elegantly des- 
cribed in the Periodic Table, an outstanding example of a model 
that provides a systematic view of matter and its interactions. 
Changes in temperature and changes in state represent a category of 
physical change among substances, within the realm of matter and 
energy. 

An understanding of the particulate nature of matter can be 
assessed through the identification of types of materials; for ex- 
ample, “mystery powders” or the equivalent at the elementary level. 
The effects of temperature and heat energy on systems are exem- 
plified in how refrigerators work, how and why ice cubes melt, and 
how metallic fuses work. Some practical areas of human activity 
(for example, cooking and much of the modem chemical industry) 
and processes (transport of materials in biological systems) are 
related both to the nature of matter and to the effects of external 
factors on its behavior. All these topics can provide rich assessment 
exercises. 

The properties of matter determine the uses to which particular 
materials are put by manufacturers, engineers, and others involved 
in technology. Materials can be physically combined or processed 
to serve human needs. Modem materials technology has focused 
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increasingly on the synthesis of materials with entirely new proper- 
ties. Chemical changes are typically involved, and the properties of 
the new materials — such as plastics and ceramics — may be entirely 
different from those of its constituents. 

The growth of technology has led to the use of some materials 
from the environment (for example, forests, ore deposits, and petro- 
leum) much more rapidly than they can be replaced by natural pro- 
cesses. There is a continuing search for substitute materials— and in 
many eases they have been found or invented. 

Disposal of used materials has become an increasing problem. 

Some used materials, such as food scraps and waste paper, can be 
returned safely to the environment — although as the population 
grows, the task becomes more difficult and expensive. Other 
materials, sUch as aluminum scrap and glass, can be recycled, with 
resulting savings in energy and resources. Some materials, such as 
plastics, are not easily recycled, nor do they degrade quickly when 
returned to the environment. Other used materials — radioactive 
waste being the most dramatic but not the only example — are so 
hazardous for such a long time that it is not clear how best to dis- 
pose of them. This issue has become the subject of widespread 
debate and controversy. Solving these problems of disposal will 
require systematic efforts that include both social and technolo- 
gical innovations. Assessment questions dealing with the scientific 
and technological issues involved in resource management are 
appropriate for grades 8 and 12. 

Energy and Its Transformations 

With respect to energy and its transformations, the 1996 and 
2000 NAEP Science Assessment should center on the following 
concepts: 

Forms of Energy — Energy Transformation (qualitative) and Audits 
(quantitative) in: % 

• Living systems 
— Plants 
— ^Animals 
— Protista 
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• Natural physical systems 

• Artificial (human-constructed) systems 
Energy Sources and Use 

• Quantity and kind 

• Y)isinh\i\\on (Patterns of Change) 

• Energy conversions, heat gain/loss, and efficiency (Patterns 
of Change, Models) 

• Slowing depletion of energy sources (conservation) 

• Costs, implications, advantages, risks, and availability 
( Patterns of Change) 

Students should understand that the concept of energy is central 
to understanding changes observed in natural and artificial systems. 
Observable changes occur when energy is added to a system, when 
energy is removed from a system, or when energy is transformed 
from one form to another. Energy appears in many forms and is 
categorized in different ways: light, heat, sound, kinetic and poten- 
tial (electromagnetic, electrical, chemical, gravitational, and 
elastic), consumable and renewable, and available and unavailable. 
Although various forms of energy appear very different, each can 
be measured in a way that makes it possible to keep track of how 
much of one form is transformed into another. 

Students need to understand that energy is conserved. Within a 
system, whenever the quantity of energy in one place or form 
changes, the quantity of energy in another place or form increases 
or decreases by a similar quantity but the total energy remains the 
same. Thus, if no energy leaks in or out across the boundaries of a 
system, the total energy of all the different forms in the system will 
not change, no matter what kinds of changes occur within the 
system. 

Energy transformations usually result in producing some 
thermal energy (heat), which “leaks away” by radiation or conduc- 
tion (for instance, from engines, electrical wires, hot-water tanks, 
human bodies, or stereo systems) and becomes unavailable for 
further transformations. Thus, the total quantity of energy available 
for transformation usually decreases. 
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Students need to develop an understanding of the more general 
principle that natural processes occur in the direction of increasing 
the total disorder of the system and its surroundings. Although 
some subsystems do increase in orderliness (such as the freezing of 
water to form ice), another part of the system or a connected system 
becomes more disordered. The cells of a human organism, for 
example, are always busy increasing order (for example, building 
complex molecules and body structure). But this occurs at the cost 
of increasing disorder even more (for example, breaking down the 
molecular structure and order of food that is eaten and warming up 
surroundings). 

Energy transformations occur both naturally and in devices 
constructed by humans. 

Naturally occurring transformations: 

• Solar energy into stored energy such as starches, fats, and 
proteins 

• Solar energy into heat 

• Potential energy into kinetic energy (for example, the poten- 
tial energy of roller coasters at the top of a hill converting to 
kinetic energy on the way to the bottom) 

Transformations occurring in human artifacts: 

• Electric mixer converts electrical energy into mechanical 
energy 

• Hair dryer converts electrical energy into heat energy 

• Automobile converts chemical energy into mechanical energy 

In the operation of these devices, as in all phenomena, the useful 
energy output — that is, what is available for further change — is 
always less than the energy input, with the difference usually 
appearing as heat. One goal in the design of such devices is to make 
them as efficient as possible — that is, to maximize the useful output 
for a given input and to minimize wasted heat energy. 

Radiant energy from the Sun is the ultimate source of most of 
the energy we use. It becomes available to us in several ways. The 
energy of sunlight is captured directly in plants, which then may be 
eaten; it also heats the air, land, and water, causing wind and rain. 



For much of history, burning wood was the most common source 
of intense energy for cooking, heating dwellings, and running 
machines. Most of the energy used today is derived from burning 
fossil fuels, which contain stored solar energy that plants collected 
over millions of years. A new source of energy is the fission of the 
nuclei of heavy elements, which — compared with the burning of 
fossil fuels — releases an immense quantity of energy in relation to 
the mass of material used. In nuclear reactors, the energy generated 
is used mostly to heat water into steam, which drives electric 
generators. 

Humans use energy for technological processes: transporting, 
manufacturing, communicating, and getting raw materials, then 
working with them and recycling them. Students need to appreciate 
that different sources of energy and ways of using them involve 
different costs, implications, and risks. Some resources will con- 
tinue to be available indefinitely; some can be made self-renewing, 
but only at a limited rate. Fuels like coal, oil, natural gas, and 
uranium will become more difficult to obtain as the most readily 
available sources become depleted. New technology may make it 
possible to use the remaining sources better; the ultimate limitation 
may be prohibitive cost rather than complete disappearance. 

Students should know that the depletion of nonrenewable energy 
sources can be slowed by both technical and social means. “Techni- 
cal means” includes maximizing the advantages realized from a 
given input of energy through good design of the transformation 
device, insulation to restrict heat flow (for example, insulating 
hot-water tanks), or additional work with the heat as it dissipates. 
“Social means” includes government, which may restrict low- 
priority uses of energy or establish requirements for efficiency 
(as in automobile engines) or insulation (as in house construction). 
Individuals may also make energy efficiency a consideration in 
their own choice and use of technology (for example, turning out 
lights and driving high-efficiency cars), either to conserve energy as 
a matter of principle or to reduce their personal long-term expenses. 
Students need to appreciate that there will always be tradeoffs. For 
example, better insulated houses restrict ventilation and thus may 
increase the indoor accumulation of pollutants. 
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The bases for these energy-related concepts should be laid in 
elementary school science. The following examples illustrate 
appropriate activities that can be used to formulate assessment 
exercises for students in grade 4: 

1. The student gives examples from his/her own experience of 
heat energy and light energy changing a system. 

2. The student identifies the source of energy for a familiar 
system (animal, plant, car, electric appliance) and describes 
some of the energy conversions that take place in each system. 

3. The student identifies the energy stored in a stretched rubber 
band and in a stretched or compressed spring as potential 
energy. The student explains that to store potential energy in a 
rubber band or in a spring, he/she must exert a force to stretch 
the rubber band, or to stretch or compress the spring. 

4. Given pictures of several situations, some of which depict a 
force being exerted or work being done and some of which do 
not, the student identifies those pictures in which a force is 
being exerted. 

5. The student explains, using the words “fuel” and “energy” in 
context, why a candle goes out when the wax is used up. 

6. The student writes a short essay on how his/her life would be 
different if all the coal and petroleum on Earth were used up. 

The Motion of Things 

With respect to this topic area, the 1996 and 2000 NAEP Frame- 
work should center on the following concepts: 

Reference Frames 

Motion 

• Force and changes in position and motion 

• Action and reaction 
Waves 

• Vibrations and waves as motion summaries 

• General wave behavior 

• Electromagnetic radiation 
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• Effects of wavelength 

• Interactions of electromagnetic radiation with matter 

What students at the three grade levels should understand about 
these concepts is summarized below; 

Reference Frames 

Grade 4 

Everything moves — bicycles, cars, and trains; the stars, planets, 
and moons; the Earth, its surface, and everything on its surface; and 
all living things and every part of living things. Positions of things 
may be described; positions may change. Monitoring changes in 
time yields information about speed. 

Grade 8 

No special point in space can serve as a reference for all other 
motion. All motion is relative to whatever point or object is chosen. 

Forces and Motion , 

Grade 4 

Changes in motion — that is, changes in speed or direction — are 
due to the effects of forces. 

Grade 8 

Any object maintains a constant speed and direction of motion 
(including being at rest) unless an unbalanced outside force acts on 
it. When an unbalanced force does act on an object, the object’ s 
motion changes. Depending on the direction of the force relative to 
the direction of motion, the object may change its speed (a falling 
apple), its direction of motion (the Moon in its curved orbit), or 
both (a fly ball). The greater the extent of the unbalanced force, the 
more rapidly a given object’s speed or direction of motion changes. 
In most familiar situations, friction between surfaces brings forces 
into play that complicate the description of motion, although the 
basic principles still apply. 
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Grade 12 

The more massive an object is, the less rapidly its speed or 
direction changes in response to any given force. Whenever 
something A exerts a force on something B, B exerts an equal force 
back on A, but in the direction opposite of the force exerted by A. 

Vibrations and Waves 
Grade 4 

Some motions can be described most conveniently in summary 
descriptions of the pattern of motion, such as vibrations and waves. 
Vibrations may set up a traveling disturbance that spreads away 
from its source. 

Grade 8 

Vibration involves parts of a system moving back and forth in 
much the same place, so the motion can be summarized by how 
frequently it is repeated and by how far the parts of a system are 
displaced during the cycle. Vibration may move through a system 
as a wave. Wave behavior can be described in terms of speed, 
wavelength, and frequency. Wavelength can_help determine how 
a wave interacts with things— how well it is transmitted, absorbed, 
reflected, or diffracted. 

Grade 12 

. Apparent change in wavelength can provide information about 
relative motion. The ways in which shock waves of different 
wavelengths travel through and reflect from layers of rock are 
important clues to the structure of the Earth’s interior. 

Light as Waves 

Grade 4 

White light is made up of all different colors of light. Things 
appear to have different colors because they reflect or scatter the 
light of some colors more than others. 
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Grade 8 

Light behaves in many ways like waves — changing direction, 
bouncing off surfaces, spreading out, speeding up, slowing down, 
and changing wavelength. 

Grade 12 

The interaction of electromagnetic waves with matter varies greatly 
with wavelength. Thus, different but somewhat overlapping electro- 
magnetic ranges have been given distinctive names: radio waves, 
microwaves, radiant heat or infrared radiation, visible hght, ultraviolet 
hght, X rays, and gamma rays. Materials that ^ow one range of 
wavelengths to pass through them may completely absorb others. For 
example, some gases in the atmosphere — including carbon dioxide and 
water vapor — are transparent to much of the incoming sunlight but not 
to the infrared radiation emitted by the warmed surface of the Earth. 
Consequently, heat energy is trapped in the atmosphere. The tempera- 
ture of the Earth rises until its total radiation output reaches a state of 
balance with the total radiation input from the Sun. 

Life Science 

The fundamental goal of life science is to attempt to understand 
and explain the nature of life. During the 20th century, the thrust of 
biological research has changed its focus from descriptive natural 
history to experimental science, with most biological investigations 
conducted within the theory of evolution. The major concepts to be 
assessed in the life sciences — with evolution as the central, unifying 
theory — are listed below and developed further in the grade-level 
descriptions that follow. 

Cells 



• Information transfer 

• Energy transfer 

• Cellular communication 
Organisms 

• Reproduction, growth, and development 

• Life cycles 

• Functions and interactions of systems within organisms 
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Ecology 

• The interdependence of life: populations, communities, and 
ecosystems 

Evolution 

• The diversity of life on Earth 

• Genetic variation within a species 

• Adaption and natural selection 

• Changes in diversity over time 

The three themes in the NAEP Science Assessment Framework 
(Patterns of Change, Systems, and Models) can be interwoven 
with these major concepts in the life sciences. Because evolution 
is the major pattern of change that occurs in the life sciences, the 
Patterns of Change theme can enhance understanding of all of the 
life science concepts listed above. Because the Systems theme can 
pertain to systems at the cellular, organismal, population, commu- 
nity, and ecosystem levels, this theme can also enhance understand- 
ing of most, if not all, of the life science concepts. Although 
Models are used in the life sciences, this theme receives less 
emphasis in life science than in the physical and Earth sciences, 
particularly at grade 4 and grade 8. 

Students understanding of life science concepts develops 
gradually as the students proceed from grade 4 to grade 8 to grade 
12. A description of the developmentally appropriate concepts that 
should be understood at each grade follows. 

Grade 4 

Organisms 

• As some animals grow, they look pretty much the same — they 
just increase in size. As other animals grow, they change from 
one form to another form that looks very different. They may 
change form several times before they become adults. 

• Only adults can reproduce, but not all young animals survive 
long enough to become adults. 
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• Many activities go on inside the body that cannot be seen. 
When something happens in one part of the body, it affects 
what goes on in other parts of the body. 

Ecology 

• Plants make their own food with sunlight, water, and air. 

• Some animals eat plants; some of these animals are eaten by 
other animals. 

• Plants and animals get energy and building materials from 
their food. 

Evolution 

• There are different kinds of plants and animals on Earth and in 
the sea. 

• There are differences among individuals of the same kind of 
plant or animal. 

• Children of the same parents are somewhat alike and some- 
what different. 

Grade 8 

Organisms 

• Different systems of the body have different functions; how- 
ever, the functioning of each system affects other systems. 

• Interactions among systems are complex. These interactions 
maintain a fairly stable operation of the entire system that can 
resist disturbance from within or without. 

• Interaction with other organisms (especially microorganisms) 
is important to maintain health or cause disease. Avoiding or 
killing microorganisms can prevent disease. 

Ecology 

• Plants use energy in sunlight to assemble food molecules from 
water and carbon dioxide. 

• Plants and animals break down food molecules to obtain food 
energy. 

• The source of energy and materials for all animals is plants. 
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• The pattern of what eats what in a community can be 
complex. 

Evolution 

• The organisms that survive long enough to reproduce may be 
different in some ways from others in a population that do not 
survive long enough to reproduce; their offspring may inherit 
the anatomical, chemical, and/or behavioral characteristics 
that enabled the parents to survive. 

• Gradually, over many generations, organisms with the favor- 
able characteristics may crowd out other organisms in the 
population that do not have these characteristics. 

• Scientists believe that these processes, operating over very 
long periods of time, have resulted in the diversity of organ- 
isms that can be seen on Earth today. 

• Adaptation may be to either the living or nonliving compo- 
nents of the environment. 

Grade 12 

Cells 

• Every cell contains a recipe for running the cell, coded in 
deoxyribonucleic acid (DNA) molecules; the code mainly 
specifies how to put proteins together. 

• During cell reproduction, the information in the DNA code is 
passed on to the next generation of cells. 

• Proteins control most of what goes on within cells and within 
the body. 

• There are interactions among the cells of an organism — 
molecules from one cell affect what goes on inside other cells. 

• In plant cells, energy from sunlight is transformed into 
chemical energy during photosynthesis; in plant and animal 
cells, the chemical energy stored in food molecules is released 
during digestion and produces heat. Some of this released 
energy is used to build new molecules. 
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Organisms 

• Separate parts of the body can communicate with one another 
using electrical or chemical signals. 

• Complex interacting systems include feedback that tends to 
produce cycles of activities within the body. 

• In organisms that reproduce sexually, each parent passes on 
one-half of its DNA information to each of its offspring; 
therefore, half of the DNA in each cell of an organism came 
from one parent, and half from the other parent. 

Ecology 

• Interactions between living and nonliving components affect 
how ecosystems function as a whole. 

• A change in one component of an ecosystem affects other 
components of an ecosystem. These components in turn 
react in a way that will restore the ecosystem to its original 
condition. 

• Often changes in one component of an ecosystem will have 
effects on the entire system that are difficult to predict. 

• The size of a population and the rate of growth of a population 
are determined largely by the survival rate, reproductive rate, 
and death rate of the organisms in the population. Predictions 
about changes in the size or rate of growth of a population can 
be described using mathematical models. 

Evolution 

• Recombination and mutation are the raw materials for new 
traits upon which natural selection acts. 

• When the environment changes, different characteristics may 
be important for survival — different adaptations are important 
for survival in different environments. 

• Some descendants are so different from other descendants that 
they can no longer breed with each other. 

Particularly at grade 12, students should be able to integrate 
information from different concepts within the life, physical, and 
Earth sciences. They should understand how key concepts apply 
at different levels of biological organization (molecular, cellular. 
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organism, population, community, ecosystem, and biome) and how 
these concepts apply to current societal problems and are signifi- 
cant to the development of a variety of biotechnologies. They 
should be able to describe common misconceptions about natural 
phenomena and to describe how these explanations ^e contrary to 
contemporary scientific explanations. Students should also under- 
stand the effects of technologies created by humans on the life 
cycles of organisms and their effects on communities of plants 
and animals, including humans. In addition, students should have 
developed some familiarity with the historical development of key 
concepts in the life sciences. 





Appendix C 

Examples of Themes by Grade Level 






Examples of Themes by Grade Level 



Systems 

S tudents should understand that systems are artificial construc- 
tions created by people for certain purposes — to gain a better 
understanding of the natural world or to design an effective 
technology. 

Understanding the construct of a system entails identifying and 
defining its boundaries, identifying its component parts, identifying 
the interrelations and interconnections among the component parts, 
and identifying the inputs and outputs of the system. 

Systems should be embedded in life science learning at the three 
grade levels in the following ways: 

Grade 4 

Systems should be approached at the level of organisms. Stu- 
dents should have broad and rich acquaintance with structure/ 
function relationships as a precursor to a more thorough knowledge 
of organ systems by grade 8. Understanding examples of food 
chains and interdependencies among organisms, say, within an 
aquarium, are precursors to understanding complex systems. 

Grade 8 

Students should understand that an organism is made up of organ 
systems that have structure/function adaptations and interconnec- 
tions among other organ systems. 

Interdependence of plants and animals in communities should be 
understood by grade 8: plants— >consumers—>decomposers. Students 
should be able to explain specific examples such as purple loose- 
strife replacing cattails and the effects of the introduction of rabbits 
into Australia. 

Disease and health should be understood in systems terms. If a 
part of a system is put out of kilter by disease, for example, the 
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whole system is affected. Taking drugs or smoking by an individual 
may have an impact on another system (organism); for example, 
secondary smoking effects on children of smoking parents or fetal 
damage from drugs. A measles vaccine taken by an individual, or 
not taken, affects the whole population of a region or further, 
depending on migration patterns. If a specific animal or plant 
population becomes unhealthy (for example, fish poisoned, rac- 
coons diseased, or species of grass infected by virus), the food 
chain and, therefore, the rest of the community are affected. 

Grade 12 

Ecosystems should be understood in their full complexity, inclu- 
ding interrelationships of plants and animals with one another as 
well as with the physical components of a system. Students also 
need to recognize the effects of human activity on ecosystems and 
the limitations on human activity imposed by natural systems. 

At this level, the cell should be understood both as a system in 
itself and as a component of a system. 

Patterns of Change 

Patterns of Change is a particularly valuable theme in the life 
sciences because a conceptual understanding of patterns of change 
can be developed in the context of several different levels in the 
hierarchy of biological organization. At the cellular/organismal 
level, the primary patterns of change are the growth and develop- 
ment that occur throughout the life of organisms. At the population 
level, the primary patterns of change are the changes in population 
growth over relatively short periods of time and the evolutionary 
changes that occur over longer periods of time. At the community/ 
ecosystem level, the primary patterns of change are those that 
involve the nonliving and living components of ecosystems during 
the process of succession. Patterns of change may be linear, or they 
may be cyclical; for example, many of the patterns of change that 
occur within cells are related to homeostasis, in which a change 
leads to feedback reactions that result in a return to conditions that 
existed before the change. An understanding of cyclical patterns of 
change can also be developed in the context of ecosystems (nutrient 
cycles) and in the context of organisms (life cycles). 
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Regardless of the context in which an understanding of the 
patterns of change theme is developed, students should be able to 
recognize patterns of similarity and difference; to recognize how 
these patterns change over time; and to transfer their understanding 
of a familiar pattern of change to a new, unfamiliar situation. 

To understand the conceptual basis for the patterns of change 
theme, students must begin to develop an understanding of major 
ideas by the 4th grade, continue to develop their understanding 
through the 8th grade, and integrate their understanding with their 
knowledge in the 12th grade. 

Grade 4 

Understanding patterns of change at the organismal level 

• Life cycles (including growth and metamorphosis) 
Understanding patterns of change at the population level 

• Concept of biotic potential, birth rates, and survival rates 

• Diversity of many types of plants and animals (an important 
preconcept for the understanding of evolution) 

• Variation within species (focus on humans, dogs, and cats) 
Understanding patterns of change at the community/ecosystem level 

• Food chains (also important for the systems theme) 

A more general understanding involves the notion that every- 
thing changes, sometimes quickly and sometimes slowly, and that 
changes may be too rapid or too slow to observe directly. 

Grade 8 

Understanding patterns of change at the organismal level 

• Growth, development, and reproduction of the human 
organism 

• Homeostasis of body systems 
Understanding patterns of change at the population level 

• Adaptation and natural selection, including learned and 
instinctive behavior 
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• Variation and similarity among many different organisms, 
including humans 

Understanding patterns of change at the coininunity/ecosystem level 

• Food webs (also part of the systems theme) 

• Environmental effects of human activity (also part of the 
systems theme) 

More general understandings involve the following knowledge: 
Changes in quantity usually have natural limits, but changes in 
form in which each form arises from a previous one can produce 
an unlimited variety; the rate of change may be as interesting as 
the change itself; and trends can be steady, accelerated, approach 
a limit gradually, or have a highest or lowest value. 

Grade 12 

Students should have acquired an understanding of the following 
concepts and developed the ability to integrate them into the 
patterns Of change theme. 

Understanding patterns of change at the cellular/drganismal level 

• Growth and development of cells, including an understanding 
of the importance of mitosis and meiosis 

Understanding patterns of change at the population level 

• Patterns of evolution, mechanisms for evolution, the conse- 
quences of evolution (such as speciation and diversity through 
time), and evidence for evolution 

Understanding patterns of change at the community/ecosystem level 

• Nutrient cycles and the impact of human activity on those 
cycles 

• Succession, both natural and as a result of human disturbance 

More general understandings entail knowing that trends, cycles, 
and randomness can occur at the same time; randomness may make 
it hard to see trends or cycles; randomness sometimes may look 
like a trend or cycle; feedback in systems — often an influence that 
reacts against change — tends to produce cycles; changes that follow 
precise rules from one moment to the next may still be unpredict- 



able in the long run; and the environment in which any one change 
occurs is usually changing also, and each affects the other. 

Models 

Models of objects and events in nature can be used as approaches 
to understanding. As such, they have limits and involve simplifying 
assumptions but also possess generalizability and, sometimes, pre- 
dictive power. Models are composed of groups of interrelated con- 
cepts selected to represent the interrelations of objects or events in 
nature or in the laboratory. Models need not be deemed Correct 
to be useful but may represent attempts to help identify relevant 
variables to build evermore useful representations. 

Models may be conceptu^ and consist of word descriptions or 
drawings. Models can also be mathematical, consisting of equations 
or other formal representations. Finally, physical models consist of 
physical objects that possess or represent some characteristics of 
the real thing. 

The solar system is often modeled conceptually in the classroom 
by describing the planets as huge balls moving about an even larger 
Sun. A mathematical model of the solar system should include 
quantitative descriptions of the gravitational forces between the 
planets and the Sun as determined by their masses and distances 
from one another, and might include the shape of a planet’s orbit as 
being elliptical. And finally, a physical model of the solar system 
might consist of a series of scale-size balls placed at appropriate 
distances throughout a room or hallway. 

Other examples from the Earth and physical sciences include 
models of shorelines and continental plates as well as stick-and-ball 
models of molecules. Physical models, such as those of the eye, 
leaf, and human torso, have been used in the life sciences for 
decades. Experiments with animals serving as models of human 
beings have been used to understand the effects of medical treat- 
ments that might be useful in preventing human diseases, and 
bacteria have been used to model population growth and decay. 

Similarly, conceptual models are common in both the biological 
and physical sciences. The simplified treatment of photosynthe- 
sis; the stages of meiosis and mitosis, accounting for an electrical 
current in a “water flow” analogy; and the characterization of gas 
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molecules as bouncing balls are examples of commonly used 
conceptual models. 

^^3-thematical models such as the gas laws and Newton's laws 
of motion are major components of the physical sciences; In addi- 
tion, some mathematical models, such as Mendel’s laws, have been 
part of the biological sciences for most of this century, whereas the 
Hardy- Weinberg formulation for describing ecosystems mathemati- 
cally has become part of introductory biological knowledge more 
recently. 

Models often serve as prototypes in technology, and in that case 
may be full-sized representations of the final product. However, 
models can be used to test the workings of technology without ' 
costly investments in full-scale objects. Small boats and airplanes 
are tested in tanks and wind tunnels before their full-size counter- 
parts are built. In this way, many experiments can be tested inex- 
pensively to optimize the design. 

Models can be easily developed as a theme and can be linked to 
the immediate experiences of children because they have grown up 
with a variety of toys. Children readily understand that most toys are 
models that look like the real objects — such as cars, airplanes, babies, 
and animals— but do not possess all the attributes of those objects. 
Many of these toys are models, sometimes scale models, of objects 
from the natural world. For example, models of dinosaurs enable 
children to develop ideas about what these creatures were like. 

The models theme has been selected because of the importance 
of enabling students to distinguish the idealizations of models from 
the phenomena themselves. Students need to understand that a 
model of the human eye does not represent all aspects of human 
eyes as they occur in human organisms. The model is a simplifica- 
tion, leaving unrepresented the many important variations in human 
eye structure, yet the simplification has utility in illuminating some 
features of the eye and enables new questions about the eye to be 
generated. 

Students need to understand the limitations and simplifying 
assumptions that underlie the varied models used in the natural 
sciences. For example, beliefs that models are replicas of “real” 
objects or events can negate the critical concept of variation that 
many models do not take into account. Although generalized 
models, such as a generalized graph of growth in populations, are 
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useful, they are not to be confused with a graph of the growth of a 
particular organism or population or with a graph of data from a 
single experiment. 

Grade 4 

At this level, models should be identified by students as repre- 
sentations of objects or events. Students can examine both concep- 
tual and physical models in terms of how they are like and not like 
the object or event being represented. Examples can be models of 
insects, seeds, leaves, and other physical objects. These models and 
others in the sciences can be linked to chil(ken’s experiences With 
scale models of cars, dinosaurs, doll furniture, and so forth. 

Grades 

Students should have knowledge of both conceptual and physical 
models and their uses and limitations. For example, when asked to 
illustrate their understanding of vertebrate structure and function 
with models of skeletons of different vertebrates, students need to 
be aware of variations in real skeletons and the generalized nature 
of the replicas. 

Grade 12 

Mathematical, physical, and conceptual models should be 
familiar to students beyond grade 8. It is appropriate to assess 
students’ ability to formalize the concept of models and their uses 
and limitations in the natural sciences and in technology. 
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Appendix D 

Science Content Outlines (Excerpts*) 








(* Complete content outlines appear in the Specifications Document) 
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Grade 4 — Earth Science 

A. Solid Earth (lithosphere) 

1. Composition of the Earth 

• Students can classify substances such as soil, sand, or rock. 

• Students can identify common geographic features of 
landscapes. 

2. Forces that alter the Earth’s surface 

• Students can describe/explain basic facts about major features 
of the Earth’s surface and natural changes in those features 
(e.g., volcanoes, glaciers). 

— Students can predict the effects of weathering (e.g., rain 
and wind on sand piles, mud piles, or rock). 

— Students can describe the relative difference in time it takes 
to erode a sand pile, a mud pile, and a rock pile ( Concep- 
tual Understanding; Patterns of Change). 

—Given a picture, topographical map or globe, or word des- 
cription of a inajor Earth feature (e.g., canyon, mountain 
range. Great Lake, cavern, or island), students can identify 
a geologic force that contributed to producing that feature 
(Conceptual Understanding; Models). 

3. Rocks: their formation, characteristics, and uses 

• Students can identify common rocks and minerals and can 
explain how we can investigate what they are made of and 
how they form. 

— Students can classify rock samples according to color, 
texture, or other identifying properties (Scientific Investiga- 
tion; Nature of Science). 

— Students can explain that molten rock comes out of volca- 
noes, hardens, and becomes part of the landscape ( Concep- 
tual Understanding; Patterns of Change). 

4. Soil: its changes and uses 

• Students know some facts about the composition of soil. 



— Students can separate soil samples into component parts 
(Scientific Investigation; Nature of Science; Systems). 

• Students recognize that plants grow in soil and that soil 
provides both nutrients and support for the plant. 

— Students can classify and relate major solid types (e.g., 
clay, sand, loam, subsoil) to their ability to support plant 
growth; that is, can identify/predict the major plant types 
likely to grow in those soils (Conceptual Understanding; 
Nature of Science). 

5. Resources from the Earth used by humankind 

• Students can identify Earth resources used in everyday life. 

Students can identify common uses of rock in the human 
environment (e.g., buildings, roads, walls) (Practical 
Reasoning; Nature of Technology). 

• Students can explain/identify that gasoline is processed from 

oil, which is pumped from the Earth (Practical Reasoning; 
Nature of Technology). 

Grade 8 — Life Science 

A. Cells and their functions 
1. Cells 

• Students can describe their observations of cells under the 
microscope. 

— Students can demonstrate the use of a microscope to 
examine a tissue, plant, or animal and to differentiate 
between plant and animal cells (e.g., students can look at an 
animal cell and a plant cell and notice that an animal cell is 
flexible and a plant cell is not) (Scientific Investigation; 
Systems). 

Students can look at pond water through a microscope and 
describe outstanding features/activities of the protista they 
see (e.g., locomotion, nutrition, excretion) (Scientific 
Investigation). 

Students can observe diatoms and try to distinguish as 
many features as possible (Scientific Investigation ). 
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• Students can explain, in a general way, the advantages of 
cellular interdependence versus independence (e.g., multicel- 
lular animals versus single-celled animals). 

• Students can describe, in general terms, the difference be- 
tween asexual and sexual reproduction in cells and the advan- 
tages and disadvantages of each [The stages of mitosis are 
not to be tested]. 

B. Organisms 

1. Reproduction, growth, and development 

• Students can describe growth, development, and reproduction 
of the human organism. 

— Students can identify the age ranges at which human beings 
go through common stages of development (e.g., can 
recognize their parents; can learn to walk, talk, socialize; 
can conceive or give birth) (Conceptual Understanding; 
Patterns of Change). 

— Students can identify the changes human beings undergo at 
puberty and can explain their functions ( Conceptual 
Understanding; Patterns of Change )_, 

— Students can, in simple terms, describe changes in human 
embryo development and the effects of environmental 
influences such as smoking, drugs, disease, and the 
mother’s diet on the development of the embryo ( Concep- 
tual Understanding; Patterns of Change). 

2. Life cycles 

• Students can identify some major influences on the human life 
cycle (e.g., diet, disease). 

— Students can discuss the influence of diet and food avail- 
ability on human life cycles worldwide (Practical Reason- 
ing; Patterns of Change). 

— Students can explain that microorganisms can cause disease 
and can identify some common diseases caused by microor- 
ganisms (e.g., bacteria, viruses, protista) [Differences 
between viruses and bacteria are not to be tested] 
(Conceptual Understanding). 



— Students can describe the immune system of animals as 
helping the animal fight disease and as controlled, in part, 
by the white blood cells in the body (Conceptual Under - ' 
standing). 

3. Functions and interactions of systems within organisms 

• Students are aware that, while different systems of the body 
have different functions, the functioning of each system 
affects other systems (e.g., students can describe/identify 
major organ systems of the human body, state their major 
functions, describe some of their interactions). 

Students can describe the primary tissues of the body and 
relate the special characteristics of each to its function 

(e.g., blood, lymph, muscle) (Conceptual Understanding; 
Systems). 

—Students can distinguish cells from other structures under 
the microscope (e.g., can distinguish between an onion cell 
and a salt crystal) (Scientific Investigation; Systems ). 

Students can describe how two or more organs of the body 
work together to perform a function (e.g., the heart and 
lungs working together in r&spimiion) (Conceptual Under- 
standing; Systems). 

• Students demonstrate an understanding of the functions and 
interactions of organ systems to maintain a stable internal 
environment that can resist disturbances from within or 
without (homeostasis). 

Grade 12 — ^Physical Science 

A. Matter and its transformations 

1. Diversity of matter (materials): Classification and types, 
particulate nature of matter, conservation of matter 

• Students can distinguish/classify objects, both regular and 
irregular, pure substances, both elements and compounds; and 
mixtures, both homogeneous (solutions, liquids, gases) and 
nonhomogeneous. 



9 



O 



92 



• Students can describe, measure, and compare substances in 
terms of mass, volume, and density/specific gravity. 

— Given a substance of unknown volume or weight and app- 
ropriate laboratory equipment, students can detennine its 
specific gravity (Scientific Investigation). 

— Given an irregular solid and the appropriate laboratory 
equipment, students can determine the density of the object 
(Scientific Investigation). 

— Students can offer a simplified distinction between weight 
and density (Conceptual Understanding). 

• Students can identify evidence that matter is composed of tiny 
particles (e.g., atoms, molecules) and that the particles are in 
motion (kinetic molecular theory). 

• Students can define, describe, and contrast physical, chemical, 
and nuclear changes in molecular terms. 

— Given various examples of changes in materials, students 
can distinguish among chemical, physical, and nuclear 
changes (Conceptual Understanding; Patterns of Change). 

• Students can discuss the conservation of matter in physical, 
chemical, and nuclear changes [Can also be tested under 
temperature and States of matter, or energy and its 
transformations]. 

2. Temperature and states of matter (physical changes) 

• Students can discuss/identify the relationship of physical 
states of matter to molecular energy. 

— Students can associate energy states with molecular motion 
(Conceptual Understanding). 

— Students can discuss/identify the energy transfers involved 
in the change of phase from solid to liquid to gas and the 
reverse [Also tested under energy and its transforma- 
tions] (Conceptual Understanding; Patterns of Change). 

• Students can discuss/identify the relationship of physical 
changes in substances (i.e., melting, boiling, thermal expan- 
sion and contraction, compression and expansion under 
pressure, increase or decrease in density) to changes in the 



structural organization of the atoms or molecules of which 
they are composed. 

Students can explain how antifreeze solutions Work to 
prevent freezing of the water in car radiators (Practical 
Reasoning; Systems). 

— Students can explain why NaCl (sodium chloride/salt) is 
added to ice when making ice cream (Practical Reasoning). 

3. Properties and uses of materials: modifying properties, 
synthesis of materials with new properties 

• Students can relate the physical properties (e.g., compressibil- 
ity, structural rigidity) of pure substances in solid, liquid, and 
gaseous states to the structural organization of particles in the 
substance and their freedom of motion. 

Students can explain/identify that the molecules in a crystal 
are arranged in a regular pattern that gives the crystal 
rigidity and causes it to take a simple geometric shape 
(Conceptual Understanding; Models). 

• Students can examine/utilize useful properties of materials. 

— Given an unknown liquid and a universal indicator with 
chart, students can determine the pH of the liquid (Scientific 
Investigation). 

— Given an unknown marking pen, students can use paper 
chromatography to identify the brand of marking pen from 
among several others (Scientific Investigation; Nature of . 
Science ). 

• Students can describe how common artificial materials are 
made, recognizing that substances can be designed to have cer- 
tain properties, and that the addition of relatively small amounts 
of some substances can significantly alter the properties. 

• Students can describe how common artificial materials are 
disposed of or recycled and can discuss the technological and 
environmental issues involved in these processes. 

4. Resource management 

• Students can discuss scientific, technological, and social 
issues involved in resource management. 
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— Students can discuss the issue of worker safety in manufac- 
turing processes that involve poisonous chemicals (Practi- 
cal Reasoning; Nature of Technology). 





Appendix E 

Consensus Committees and Project Staff 




96 



NAEP Science 
Consensus Project Team 



staff 

Ramsay W. Selden 
Director 
State Education 
Assessment Center 
Council of Chief 
State School Officers 

Richard C. Clark 

Consensus Coordinator 
Council of Chief 
State School Officers 

Julia H. Mitchell 

Associate Research Scientist 
American Institutes 
for Research 

Senta A. Raizen 
Director 

National Center for Improving 
Science Education 

Bonnie L. Verrico 

Administrative Assistant 
Council of Chief 
State School Officers 

Steering Committee 
Members 

William O. Baker, Retired 

AT&T Bell Laboratories 
Murray Hill, New Jersey 



Mary Louise Bellamy 
Education Director 
National Association of 
Biology Teachers 
Reston, Virginia 

Frank Betts 

Director 

Curriculum Technology 
Center 

Association for Super- 
vision and Curriculum 
Development 
Alexandria, Virginia 

William B. Campbell 

Executive Director 
National Industry Council 
for Science Education 
College Park, Maryland 

Glenn A. Crosby 

Professor 

Chemistry Department 
Washington State University 
Pullman, Washington 

Gerald Difford 

Executive Director 
Colorado Association 
of Schools 

Englewood, Colorado 



87 

99 



Janice Earle 
Director 

Center on Educational Equity 
National Association of State 
Boards of Education 
Alexandria, Virginia 

John Fowler 
Director 

Triangle Coalition for 
Science and Technology 
College Park, Maryland 

Johnnie Hamilton 

Principal 

Franklin Intermediate School 
Chantilly, Virginia 

Elam Hertzler, Retired 

Secretary’s Commission on 
Achieving Necessary Skills 
Washington, D.C. 

Ann Kahn, Former President 

Parent Teachers Association 
Fairfax, Virginia 

Douglas Lapp 
Director 

National Science 
Resources Center 
Smithsonian Institution 
Washington, D.C. 

John Layman 
Director 

Science Teaching Center 
University of Maryland 
College Park, Maryland 

Harold Pratt 

Executive Director 
Science and Technology 
Management 
Jefferson County Scho^s 
Golden, Colorado 9 S 



Douglas Reynolds 

Chief 

Bureau of Science Education 
State Department of Education 
Albany, New York 

Bella Rosenberg 

Assistant to the President 
American Federation 
of Teachers 
Washington, D.C. 

Jane Sisk 
Biology Teacher 
Calloway County High School 
Murray, Kentucky 

Judith Torney-Purta 

Department of Human 
Development 
University of Maryland 
College Park, 'Maryland 

Planning Committee 
Members 

Andrew Ahlgren 

Associate Director 
Project 2061 

American Association for the 
Advancement of Science 
Washington, D.C. 

Bill Aldridge 

Executive Director 
National Science Teachers’ 
Association 
Washington, D.C. 

J. Myron Atkin 

Professor 

School of Education 
Stanford University 
Stanford, California 



Joan Boykoff Baron 

Assessment Coordinator 
Connecticut Common Core 
of Learning 

Bureau of Evaluation and 
Student Assessment 
Connecticut Department 
of Education 
Hartford, Connecticut 

Audrey Champagne 

Professor 

School of Education 
State University of New York 
Albany, New York 

Sally Crissman 

Lower School Head/ 

Science Teacher 
Shady Hill School 
Cambridge, Massachusetts 

Edmund W. Gordon 

Professor 

Department of Psychology 

Yale University 

New Haven, Connecticut 

Henry Heikkinen 

Director 

M.A.S.T. Center 
University of 
Northern Colorado 
Greeley, Colorado 



George Hein 

Professor 
Lesley College 
Cambridge, Massachusetts 

Joseph L. Premo 

Science Consultant 
Minneapolis Public Schools 
Minneapolis, Minnesota 

Senta A. Raizen 

Director 

National Center for 
Improving Science Education 
Washington, D.C. 

James Robinson, Retired 
Curriculum and Evaluation 
Boulder Valley Schools 
Boulder, Colorado 

Thomas P. Sachse 

Manager 

Math and Science Unit 
California Department of 
Education 

Sacramento, California 

Gary E. Skaggs 
Test Development Analyst 
Office of Research and 
Evaluation 

Fairfax County Public Schools 
Falls Church, Virginia 



99 



101 






NOTES 












100 




U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 
National Library of Education (NLE) 
Educational Resources Information Center (ERIC} 




NOTICE 

REPRODUCTTON BASIS 



□ 



This document is covered by a signed “Reproduction Release 
(Blanket) form (on file within the ERIC system), encompassing all 
or classes of documents from its source organization and, therefore, 
does not require a “Specific Document” Release form. 



r~j/ This document is Federally-funded, or carries its own permission to 
reproduce, or is otherwise in the public domain and, therefore, may 
be reproduced by ERIC without a signed Reproduction Release form 
(either “Specific Document” or “Blanket”). 




EFF-089 (9/97) 





